ab9801519016f991e20f8b10d7704f068fd02564
15
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 John O'Keefe
|
ab98015190 |
feat(collections): RLS ownership enforcement at the DB layer
collections + collection_items join the RLS-governed tables, same doctrine as the media_items family lens (2026-10-29). Policies route through two PURE predicate functions (app_can_read_collection / app_can_write_collection) that never query collections — no policy recursion under FORCE RLS — so a future sharing model is one EXISTS branch inside the functions, not a schema re-touch. Reads: owner or admin; writes: owner, admin escape for SYSTEM collections only. Why: the write path had no app-level ownership gate on Update/Delete/ AddBooks — any authenticated user could mutate another user's collection by id. The DB is now the enforcement of record; existing app-level gates (match-mode, reorder, view-settings) stay as defense-in-depth. Behavior changes (contract_pins 2026-10-31): - cross-user GET detail: 200 -> 404 (row invisible) - cross-user PUT update: silent success -> 404 (ErrNoRows mapping; service wrap fixed %v -> %w so errors.Is sees pgx.ErrNoRows) - cross-user POST books: silent success -> 404 - cross-user DELETE: 204, now a true DB no-op - anonymous sees nothing; admin persona unchanged Bootstrap: registration seeds system collections BEFORE the JWT exists — CreateDefaultCollectionsForUser declares the new user's identity (rls.WithUser), the same fix mirrored in the test harness (createDefaultCollectionsForUser owner-scoped; collection sweeps run under the owner's identity). Tests: new TestCollectionsRLS pins the cross-user matrix end-to-end (404 detail/update/add/view-settings, 204 no-op delete, admin escape, owner positive control, registration seeding under RLS). Harness fixture writes to non-system collections now run as the row's OWNER (scopeExecAs/ownerSeedCtx) — the admin write-escape covers system rows only, and bare-ctx pool reads of collections see nothing. Filter/playlist/audiobook/scan-state fixtures converted. Full container suite green (159s). |
||
|
John O'Keefe
|
cc0e9d7f91 |
feat(podcasts): the episodes authoring door + curated chain order
"Newest episodes from my selected podcasts" is now buildable in the UI, end to end — the user could not find ANY podcast-related filter surface because none existed: the New Collection modal could only create book-scope smart collections, and the engine's episodes scope (item 3.1) had no authoring door. The scope door: - The create modal's Smart card gains a Rules-scope segmented control (📚 Books / 🎧 Podcast episodes); the copy no longer claims smart means "match books by rules". Choosing Episodes seeds an EMPTY episodes tree ({"scope":"episodes","mode":"all","children":[]}) so the collection is born episode-scoped: the editor opens with the episode vocabulary (feed/downloaded/played/published date) and membership runs the episode engine from the first rule. - ParseFilterTree now accepts an EMPTY ROOT (nested empty groups stay invalid) — the seed's legal shape. The root arrives at depth 1 by call convention, hence the depth>1 exemption. - The composers define the empty-tree semantics: zero children matches NOTHING (ComposedLiveMedia/ComposedCount guards), mirroring EvaluateSmartCollection's zero-enabled-rules convention. Without the guard the composed query degrades to the bare scope head and a fresh collection would swallow every episode in scope — pinned by TestEpisodeCollectionsEmptySeedMatchesNothing (envelope book_count 0) and TestEmptyRootTreeSeedsEpisodesScope. The curated chain order (plan §10 R2, user need: the briefing plays NEWEST-first): - view_settings.chain_order ("native"/"newest"/"oldest"/"reverse") stored on the collection — curated content, not a player-side knob. - PUT /api/collections/:id/view-settings merges the key into the stored JSONB (a blind write would clobber the playlist toggles). - buildSmartCollectionSequenceContext routes OrderEpisodes through handlers.CollectionChainOrder; the detail header's Play-as-Sequence gains the order select next to it (PlayCollectionButton), which PUTs and reloads. Pinned by chain_follows_the_curated_order (reverse flips the chain indexes and hops). Caught by the new test, a latent clobber fixed: UpdateCollection treated a NIL view_settings payload as "write NULL" — any edit that carried no settings dropped every stored key (chain_order, playlist toggles from API clients, imported provenance). Nil now means untouched, the same convention filterTree already had. Composer guards also cover ComposedCount, so the builder's live count reads the same semantics. Tests: collectionChainOrder unit matrix, empty-root parse unit pin, two integration pins (seed matches nothing; curated order drives the chain and survives whole-map edits). |
||
|
John O'Keefe
|
6b2125a45a |
feat(podcasts): smart collections ride the unified filter engine — filter_tree column (item 3.2)
collections gains filter_tree JSONB (idempotent in-place; same qf DSL as saved filters — the scope key included). IsCollectionSmart is the ONE detection (tree OR legacy rules), and EvaluateSmartMembership is the ONE evaluator: tree-bearing collections run the composed engine via ComposedLiveMedia (full media rows, so the collection surfaces keep their episode-card stamping, feed positions for the replay chain, and the family lens rides the mandatory head), while legacy rule arrays keep per-item FieldMatches until rewritten — one call surface, two storage epochs. All smart surfaces (detail JSON, counts, detail page, reader chain) dispatch through it; the search service is injected into the collection subsystem (variadic — main.go builds one shared instance; the test harness mirrors it). Write path: CreateCollection/UpdateCollection accept the tree, validated against the engine grammar at the HANDLER (an invalid tree is a 400, never a stored render-time 500) and exposed in the envelope (pin widened in TestContractPin_CollectionsJSON + changelog). Update semantics: an edit form that doesn't send a tree keeps the stored one. RulesToFilterTree converts legacy rules + match mode at upgrade time: episode rules map mechanically (feed→in-list, played/downloaded→ booleans, published_within→published_date+within_days), tree root mode absorbs rule_match_mode, episode rules force the episodes scope, and legacy starts_with/ends_with fall back to contains (documented). Read-compat by design: nothing is migrated at boot — legacy arrays evaluate exactly as before until the collection is rewritten. Pinned by TestRulesToFilterTree (unit conversion) + TestEpisodeCollectionsFilterTree (a tree collection evaluates through the engine, the chain follows the tree's live membership, invalid trees 400 at write, the tree wins when legacy rules coexist, and an edit without a tree keeps it). |
||
|
John O'Keefe
|
abb7c84833 |
feat(podcasts): smart episode collections — ALL/ANY match modes + episode rule fields (item 2.4)
The daily briefing needs AND: today's rules engine was OR-only. The
match semantics become a per-collection column
(collections.rule_match_mode, 'any' legacy default | 'all'), flipped on
its own endpoint (PUT /api/collections/:id/match-mode, owner-only —
nothing else on the row changes) and surfaced in the rules page as an
ANY/ALL toggle with matching copy.
Episode rule fields join the vocabulary (plan item 2.4): feed (the
multi-select rides the new `in` operator — comma-separated feed ids),
played (the canonical finished predicate — viewer progress ≥ 0.999,
same as hide_finished), downloaded, published_within (N days), plus
title/description contains. The rules candidate query now carries the
viewer's progress (LEFT JOIN reading_progress) so played/unplayed is
per-user, and TestRules/preview speak the SAME evaluator: the handler's
duplicated field switch is deleted and delegates to
CollectionService.FieldMatches — one vocabulary, one semantics.
Legitimate semantics note: in ANY mode `played:false` matches unplayed
BOOKS too (everything with no progress row) — correct per-rule
behavior, and exactly why the briefing mode exists.
Backward compatibility: every legacy row defaults to 'any' (the old
OR), UpdateCollection keeps the stored mode when the form doesn't send
one, and the evaluator treats absent data honestly (no date → no
match; no progress → unplayed).
Pinned by TestPodcastSmartCollections: the AND-mode briefing
(feed∈{A} ∧ unplayed ∧ <7d = exactly the new episode), OR-mode
widening, owner-only flips, the preview honoring the mode, and
downloaded/contains rules. Also fixes a latent bug the new query
surfaced: UpdateCollectionViewSettings referenced a nonexistent
updated_at column (never exercised until now — the new sibling query
was written without it).
|
||
|
John O'Keefe
|
8631aa00bf |
refactor: remove dead code across server, templ pages, and client modules
Deadcode-verified removals (RTA reachability from main, cross-checked with -test and textual xrefs): Go server: - whole files: middleware/transaction.go, middleware/error_handler.go, middleware/password_validator.go (live registration machinery kept), handlers/sync.go, sync/offline.go - unrouted endpoints: auth UpdateUsername/UpdateEmail, timezone settings handler, OPDS RegisterOPDS, Kobo LibrarySync handler, ReaderHandler.ShowReader, JSON-data twins on collections/library - duplicate book-matching handlers (book_matching.go Handler twins, matching.go MatchingHandler twins), superseded helpers (router.parseUUID, handlers.ValidateDeviceToken, normalizePath, userExists, generateLockID) Templ/client: - never-routed pages: api_explorer.templ, queue.templ and their client modules api-explorer.ts, api-explorer-docs.ts, queue.ts - fully orphaned wrapper modules events.ts, dom.ts - broken clearSyncQueue (called a nonexistent endpoint) and the hidden, never-populated sync-queue section on the devices page - 166+ unused exports demoted to private (knip-verified); ~85 truly dead declarations removed via tsc noUnusedLocals fixpoint - stale 'interactive API Explorer' claims dropped from docs The docs drift-gate package (internal/docs), templ-bound modules (dashboard, stats, header, index, login, profile*, bookshelf, collections, toast-error), sqlc-generated code, and web/static are intentionally untouched. |
||
|
John O'Keefe
|
f94e7472b4 |
feat(filters+ui): per-element author/publisher filters, playlist toggles, Up Next API, reorder UI, dashboard badges
Phases 4 + 5 of the metadata overhaul (plan §4, §5) plus §3 handler wiring. Filters (§4.1): author_filter/publisher_filter now match per-element against the authors[]/publishers[] arrays — a co-authored book matches any one of its authors — with comma-joined params acting as OR-within-facet multi-select (string_to_array + cross unnest). Legacy single-value params unchanged on the wire; series/genre/language filters stay on their varchar columns; narrator filter already per-element. Collections API (§3.3-3.5): collection detail carries viewer + hide_finished into the listing query and exposes up_next (visible- library-enforced reading-queue head); PUT /api/collections/:id/items/order (owner-only, full-order rewrite); UpdateCollectionRequest gains hide_finished/show_up_next ([] string last-value-wins for the hidden-false-sibling checkbox pattern) merged server-side OVER stored view_settings so imported_from_series provenance survives edits; EditCollectionModal renders the toggles from view_settings. Dashboard (§5.5): sections JSON carries is_playlist/complete (dashboard.ts is the AGENTS-documented JSON-render exception; first paint stays templ). Fully-read playlists render '✓ Playlist complete' instead of an empty rail; imported playlists get a 📖 Playlist badge. Frontend: book detail shows subtitle + full authors + Explicit badge; edit modal gains subtitle, comma-separated authors/publishers (client- split to arrays exactly like narrators), step=any series number, explicit checkbox; collection book cards get playlist move ↑/↓ buttons persisting the full order via the reorder endpoint (templ partial re-render on failure — DOM order is client-local, API order is truth). AGENTS.md gotcha #3 amended per plan §3.7: user collections are dashboard sections by default; the show_on_dashboard premise is retired. Verified: full affected integration sweep green (pins widened for up_next/is_playlist/complete — changelog entries in docs/contract_pins.md); unit suite green; go build + templ + vite green. |
||
|
John O'Keefe
|
1dbb0ce406 |
feat(collections): playlist auto-creation, position ordering, hide_finished, Up Next, dashboard eligibility
Phase 2b + 3 of the metadata overhaul (plan §2.4, §3.1-3.7). Auto-playlist collections (§2.4): the scanner turns obviously-playlist sidecar series (explicit '(Reading Order)'/chronological/publication/ in-order/timeline markers only — precision over recall) into curated static collections: upsert by view_settings.imported_from_series then name (rename-safe), position set ONLY on insert (clean integers; 1.5/1a stay unpositioned rather than guessing), membership add-if-missing, never removed or re-ordered by rescans, provenance in description + view_settings, show_on_dashboard true (dashboard-eligible by design), 'scanner.auto_playlist_collections' setting (default on, Settings Registry + schema seed). Ordering (§3.1-3.2): collection listings order position ASC NULLS LAST then added_at ASC (was added_at DESC — walked playlists backwards); PUT /api/collections/:id/items/order applies full-order rewrites (owner-only, service-gated). hide_finished + Up Next (§3.3-3.4): GetCollectionItemsForDashboard gains viewer_user_id + hide_finished (LEFT JOIN reading_progress, COALESCE percentage < 0.999) and viewer_percentage rides the row for card graying; GetCollectionUpNext serves the collection's reading-queue head with the family-lens visible-library array enforced. Dashboard (§3.5): user collections are dashboard-ELIGIBLE BY DEFAULT — the show_on_dashboard gate (never writable for user rows since the column shipped) is gone from GetUserCollectionsForDashboard; hiding stays dashboard-prefs (hidden_collections) territory; hide_finished filtered-everything now emits a Complete section instead of silently vanishing; sections JSON carries is_playlist/complete. Series page ordering hardening (§3.6): GetSeriesBooks tiebreaks by title ASC — NULL/tied numbers previously returned heap (scan) order, which is what the user saw. Frontend custom-section-builder note: the builder has sent show_on_dashboard: true since its creation; the API never bound it. User-row visibility is now intentionally prefs-governed, so the field stays unbound by design (documented in code). Pins widened (docs/contract_pins.md): collection detail + up_next; dashboard sections + is_playlist/complete; new reorder endpoint recorded. Verified: TestPlaylistCollectionsEndToEnd (new, full e2e: sidecar series split, collection auto-creation with position 7, rescan idempotency, owner-only reorder, finished-filter + completion marker + Up Next drain) plus the full affected integration sweep green. |
||
|
John O'Keefe
|
18727fef93 |
refactor(collections): rule evaluators take the real row; both 45-field converter twins deleted
checkRulesAgainstBook/evaluateRule and the service's EvaluateRules existed in two parallel universes: one over database.MediaItems (what every query actually returns) and one over ListMediaItemsRow (what two hand-written 45-field mediaItemsToListMediaItemsRow converters produced — verbatim twins in the handler and the dashboard service, with a URL-resolution side effect baked into a "converter"). The rule fields (genre/series/author/language/publisher/copyright_year/tags) all exist on MediaItems, so the evaluators now take the real row and the converters, their three call-site loops, and the dead ApplyAutoAssignRules entry point (zero callers) are gone. The rules surface is wire-covered by the collections rules scenarios; handlers+services suites and the full pin/H1/H3 battery green. |
||
|
John O'Keefe
|
6457fbaf72 |
feat(stats): reading statistics — session capture wired + /stats page + widget + per-book card
The design parked in the handoff (multi-tier stats off reading_history; bookorbit-style in data, not looks) is in. The real prerequisite work was data: reading_history existed but NOTHING wrote to it — the whole stats feature was sitting on an empty table. - CAPTURE (no client changes, no repo-boundary crossings): every successful progress write through SaveProgress (KOReader + app + web + Kobo + offline queue — the one chokepoint) now records one reading_history session row: span = the interval since the PREVIOUS write instant, pages = positive page delta when both sides know the page index, marker rows for first contact. Spans over 2h are kept as zero-second markers (overnight suspend→resume, offline flushes must not inflate totals) — the honest 'sync cadence, not a stopwatch' caveat stays. device_id only when a REAL device row matches (the web/app progress path historically stuffs the USER id into DeviceID — probe instead of trusting it; FK violation would silently drop every capture). Source preserved in device_metadata. - API: /api/stats/summary?range=7|30|90|year (totals, zero-filled daily arrays, streaks, most-active day/weekday, top books, library breakdown — one call), /api/stats/this-week (widget: rolling 7d vs previous + streak), /api/stats/books/:id (lifetime + recent sessions). All bucketed in the user's timezone; finished ≡ any push ≥0.995; visible-library lens enforced at source. Legacy /api/analytics/* kept untouched. Range windows anchor end-of-day to the START of today (blasting the clock time into tomorrow resolved 'today' a calendar day late — found by the streak test). - /stats page (sidebar Stats between Progress and Devices, previously /analytics was an orphan): range selector, five stat cards, two Chart.js v4 charts (daily minutes bar, pages/day line — chunked via dynamic import so main.js stays untouched), per-library breakdown, most-read list, empty-state explainer. /analytics now 301s to /stats. The orphaned analytics.templ + analytics.ts removed (their containers and the page JSON never matched; the page was astub). - Dashboard 'This Week' widget (server-rendered, suppressed on service failure) and the book detail 'Your Reading Stats' card (lifetime totals of the viewer's own sessions) complete the tiers: per-user default, per-library breakdown inline, per-book on the page, per-system/admin aggregate parked as before. Tests: statistics_test.go (capture semantics at the chokepoint, tz/day buckets, zero-fill, streak math, finished detection, range validation, auth gate, widget, per-book lifetime). Full suite PASS. |
||
|
John O'Keefe
|
0a7fbd6f48 |
feat(collections): smart (dynamic) collections — rules evaluated live at render
Type is DERIVED, not stored: a saved ruleset makes a collection SMART,
no rules = STATIC (stored collection_items links, unchanged behavior).
No schema change — collections.auto_assign_rules JSONB already carried
the config; user's decision 2026-10-14 (handoff: live-at-render, not
refresh-on-scan; create-time Static/Smart toggle; v1 NO hybrid).
- Rule gains an enabled flag (absent = enabled, legacy rules unaffected).
- Rule CRUD: POST/PUT(DELETE) /api/collections/:id/rules[/:ruleId] — the
survivor dead effort of the rules-builder UI finally has real
endpoints (rules were never editable after creation; nothing ever
re-evaluated them). First rule converts a user collection to smart;
removing the last reverts it to static. UpdateCollectionRulesJSON
edits ONLY the rules doc so rule CRUD cannot clobber metadata.
- Live evaluation: candidates from GetVisibleLiveMediaForRules —
visible-library set (family lens enforced at source) + archived IS
NULL AND missing_scan_count = 0 — any enabled rule matches, title
order. Zero refresh hooks by design: always current by construction.
- Membership mutations (add/remove/bulk-add) are 400 on smart — v1 no
hybrid; collection_items.excluded stays the reserved future hook.
- Rules builder page (/collections/:id/rules) first time ever routable;
templ whole-attribute interpolation ("{ expr }" inside a quoted attr
value is literal text, as theme-{ user.Theme } silently proved).
- UI: create modal Static/⚡Smart cards (smart create redirects into the
rules builder), ⚡ badge + real per-card counts on the grid (smart =
live matches, static = stored-link count), smart detail renders
matched books with no hand-pick actions + an Edit Rules button.
- API: GET /:id gains is_smart + live books; GET /: keeps the Android
app's default exclusion, include_auto=true surfaces smart rows with
live counts; TestRules previews against the SAME candidates the live
render uses (visible + live, was raw first-1000).
- FIXED pre-existing regression from
|
||
|
John O'Keefe
|
7ce73213ae |
feat(server): system collections become permanent — soft-delete + in-place restore + query_type-keyed provisioning
The old model hard-deleted system collections and re-created them at the tail, resurrecting stale dedups and breaking ordering. Redesign (user-directed B-final): - schema.sql: collections.deleted_at TIMESTAMPTZ (idempotent ALTER — the whole schema is re-applied on every boot via executeSchema). - queries.sql: every collection list/read query filters deleted_at IS NULL; new GetSystemCollectionByQueryType :one (unfiltered, sees tombstones), TombstoneSystemCollection :exec, ClearCollectionItems :exec, RestoreSystemCollectionRow :one (in-place metadata reset + un-tombstone; collections have no updated_at column — ResetUpdatedAt was a 500 caught by tests). sqlc regenerated. - collection_service.DeleteCollection: system collections are tombstoned (preserving position/ownership); user collections keep the hard delete. No ErrNoRows to the client (delete is idempotent). - dashboard_service.RestoreSystemCollection: resolve by query_type; found → in-place restore (ClearCollectionItems + metadata reset, un-tombstoned); missing → provision fresh. "Want to Read"/etc. stay at their home positions forever. - handlers/collections.CreateCollection: name-taken is 409, not 500. - handlers/auth.CreateDefaultCollectionsForUser: idempotent by (user_id, query_type) — update-in-place when a row for that query type exists (heals renames/tombstones), insert when missing; this also fixes the duplicate-system-collections regression at source. |
||
 john-okeefe
|
e1aef8e85f |
refactor(services): Modernize Go code style in collection and filters services
Apply Go 1.18+ language features and modern style:
internal/services/collection_service.go:
- Use map[string]any instead of map[string]interface{} (Go 1.18+)
- Use range clause with single variable for iteration-only loops
- Replace if-else chains with switch statements for better readability
- Remove explicit type initialization for zero values
internal/services/filters.go:
- Add Err prefix to custom error variable for error naming convention
internal/router/library.go:
- Use cfg.ProcessingIssuesHandler instead of local processingIssuesHandler variable
- Ensures proper dependency injection through router config
These changes follow current Go best practices and improve code readability.
|
||
|
john-okeefe
|
516cec5a7f |
feat: migrate tags and contributors from TEXT to TEXT[] arrays
Convert tags and contributors columns from comma-separated strings to PostgreSQL TEXT[] arrays for better data normalization and query performance. Database Changes: - schema.sql: Change tags/contributors from TEXT to TEXT[] - schema.sql: Add GIN indexes for fast array searches - queries.sql: Update search queries to use ANY() operator - queries.sql: Update fuzzy search with unnest() for arrays Generated Code (sqlc): - models.go: Auto-generated with []string types for tags/contributors - queries.sql.go: Auto-generated with proper array handling Handler Changes: - media.go: Update request structs to use []string for tags/contributors - media.go: Remove pgtype.Text wrapping, use direct array assignment - media.go: Add tag normalization in CreateMediaItemHandler - collections.go: Update tags evaluation to join arrays for comparison - collections.go: Add strings import for Join() function Service Changes: - ebook_scanner.go: Update EbookMetadata struct to use []string - ebook_scanner.go: Remove string Join(), assign arrays directly - collection_service.go: Update tags rule evaluation to join arrays - collection_service.go: Add strings import New Utilities: - internal/utils/tags.go: Create NormalizeTags(), JoinTags(), SplitTags() - Normalizes tags by trimming, lowercasing, removing duplicates/empties API Documentation: - bruno/media-items/Create Media Item.bru: Update examples to use arrays - bruno/media-items/Update Media Item.bru: Update examples to use arrays - Update docs: tags/contributors now array of string Breaking Change: - JSON format changes from "tags": "tag1,tag2" to "tags": ["tag1", "tag2"] - Tests already use array format (no changes needed) Benefits: - GIN indexes enable faster array searches - Normalization prevents data quality issues (case, duplicates) - Array operations use PostgreSQL native operators (ANY, &&, unnest) - Better separation of concerns (no string parsing in application) |
||
|
john-okeefe
|
00a083b60b |
Rename backend code references: Bookmann → Bookhoard
Backend changes: - Update import paths: bookmann/internal → bookhoard/internal - Rename struct fields: BookmannUUID → BookhoardUUID - Update handler function names: mapContentIdToBookmannUUID → mapContentIdToBookhoardUUID - Update HTTP response headers: X-Bookmann-* → X-Bookhoard-* - Update service and middleware references - Update main.go imports and references This is part 2 of the project rename to Bookhoard. |
||
|
john-okeefe
|
964a4583ab |
feat(services): Add book matching and collection services
- Add book matching service for intelligent book deduplication - Add collection service for collection management - Add test files for book matching and collections |