adb93cba9eb889ef1ecc53e54d6a86700475ad56
378
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 John O'Keefe
|
adb93cba9e |
feat(podcasts): episode collections — picker scope, hint rows, the chain (item 2.3)
D11 lifts: podcast episodes (media_items rows) join collections, and "Play collection as sequence" feeds the player from the collection's order. No new tables — collection_items already carries ordered membership (item 2.1 audit). PICKER: the collection modal gains a Books/Episodes tab pair. The episodes tab hits /api/media-items/search?scope=episodes — a dedicated picker query (SearchEpisodesForPicker) whose visibility is entirely RLS: the media_items subscription predicate drops unsubscribed feeds' episodes, and the podcast_feeds join is governed the same way. The books scope keeps its podcasts exclusion (plan §9) — this scope is the only door in. Pending episodes are pickable (plan ahead; the detail page shows them as not downloaded). Selection rides the existing bookPicker store — ids never collide across scopes, and AddBooks works unchanged on episode ids. DETAIL: the All-Libraries membership view switches to GetCollectionItemsDetailed — LEFT JOIN + per-row visibility. An unsubscribed episode (or family-lens hidden book) survives as a Hidden card rendered as an "Unavailable — remove entry" hint row instead of silently vanishing; visible episodes render feed title, download state, and a Play link. The library-scoped view keeps its existing dashboard-query behavior (the switcher's contract is untouched for books). CHAIN: ?queue=collection:<id> on any reader URL makes buildPodcastSequenceContext produce the COLLECTION context (§10 R2 same shape, source=collection): downloaded episodes in collection order (positions are content — R1; no feed-order override), prev/next URLs carrying the queue param so the chain survives hops, and backToDetail landing on the collection page. Ownership gates the chain — foreign/deleted ids fall back to the feed queue; the order control stays hidden (2.2's source gate). The header's "Play as Sequence" button starts from the first downloaded episode in list order. Pinned by TestPodcastEpisodeCollections (7 subtests): RLS-scoped picker + family-lens auditor case (subscribed episode in a hidden library must not reach the picker), books-scope exclusion intact, hint rows, chain carrying the queue param, start-from-first, foreign fallback, and reorder driving the chain (the R4 primitive works on episode ids unchanged). |
||
|
John O'Keefe
|
f47734dd7d |
fix(podcasts): generic media delete refuses episode rows (invariant 2)
The admin endpoint DELETE /api/media-items/:id predated podcasts and would delete any media row — including podcast episodes — if hit directly. Invariant 2 (absolute form, user decision 2026-10-30) says episode rows are removed ONLY by whole-feed removal: "the feed should match the XML until the entire feed is gone." The plan audit found this endpoint was the one remaining surface that could violate it: the poller, unsubscribe, and the admin file-deletion endpoints are files-only, and the hash-conflict merge can't see episodes (they never hash). The guard sits right after the GetMediaItem read and BEFORE the storage-down check, so a refused episode never touches storage resolution. Same segregation pattern as the generic edit path (D9/D10, UpdateMediaItem). Pinned by TestDeleteMediaItemPodcastGuard: admin DELETE of an episode → 409 with the row surviving; a book delete through the same endpoint → 204 control. |
||
|
John O'Keefe
|
7d5bbd1177 |
feat(admin): multi-select episode file deletion — checkboxes on downloaded rows
The plan's per-EPISODE bulk select for the admin view (§0 item 8 follow-up — trimmed in the first admin-view commit; user called it back). Admins now tick checkboxes on DOWNLOADED episode rows and delete their files in one request: - Checkbox per row, rendered ONLY for admins + downloaded episodes (nothing to delete otherwise), via the existing podcastDetail Alpine component: selection ids/count survive the list's htmx swaps (reapplied on afterSwap — same pattern as the text filter) and clear after a successful bulk delete. - Toolbar segment outside #episode-list (survives swaps): 'N selected' + 'Delete selected files' (hx-include gathers the CHECKED boxes); disabled-looking at zero selection; hx-confirm states the re-materialization contract. CSP-safe method bindings only (bulkSelectionLabel/Class) — no ternaries in Alpine expressions. - POST /feeds/:id/episodes/delete-files (admin): repeated episode-file-select keys parsed via ParseForm directly (never bound — repeated-key wire is another echo-bind gotcha); each id validated to belong to THIS feed, so a forged foreign id is refused while the real selections apply; rows reset to pending, files removed. - Tests: bulk of two real ids + one forged id → deleted:2, files gone, rows pending; non-admin 403; all prior single/feed-level pins hold. |
||
|
John O'Keefe
|
0741e98ea5 |
feat(admin): all-podcasts view — disk-first maintenance for every feed
§0 item 8 (Phase-4 item 1 pulled forward, user directive: maintenance
tooling ships WITH the features it maintains):
- /admin/podcasts (AdminMiddleware): every feed from EVERY user via the
RLS admin escape (first-admin identity, poller/fetcher contract) —
feed + host, library, subscriber count/names, downloaded/total, DISK
USAGE, last fetch. Sorted disk DESC ('what's eating the disk'),
totals header (feeds · episodes · bytes on disk).
- Per-feed actions: Refresh · Delete files (rows → pending, disk freed;
confirms spell out the re-materialization contract) · Remove feed
(rows + files for everyone — the same admin delete as the detail
page). All confirms ride the global confirm modal.
- New sqlc: ListAllPodcastFeedsAdmin (correlated aggregates),
ResetPodcastEpisodeFileDeleted / ResetPodcastFeedFilesDeleted —
downloaded rows reset to pending with download-derived facts cleared
(audio_metadata, chapter_count, cover, size, mime); file_path KEPT
(fetch-maintained layout — a re-download lands on the same path).
file deletion via the existing DeleteFeedFiles file-removal pattern
(episode files + .cover.jpg siblings).
- Poll upsert never touches download_state, so deleted files STAY
deleted — a feed refresh re-lists metadata but re-materializes
nothing (pinned by TestAdminDeleteEpisodeFiles).
- Per-episode admin delete-file: POST /feeds/:id/episodes/:eid/
delete-file (admin) — one episode's file gone, row pending, siblings
untouched; HX swaps the episode list. (Found while testing: the
handler first read c.Param("id") — the FEED's uuid — instead of
:eid; the 404 'episode not found' trail led straight to it.)
- Tests: admin sees other users' feeds + subscriber names; regular
users 403 on page AND actions; single + feed-level deletion reset
rows/clear facts/delete files; refresh leaves pending rows pending.
- Admin nav gains 'Podcasts' under Administration.
Wrong-gotcha note for future agents: parseUUID on c.Param('id') inside
a /feeds/:id/episodes/:eid route resolves the FEED, not the episode.
|
||
|
John O'Keefe
|
5c545534c2 |
feat(podcasts): Download All queues in the LISTENING order + add-modal order dropdown
Download All previously queued in feed-native order — 'getting the last chapter first doesn't help me'. Now the queue follows the feed's effective listening order (§0 item 5b): QueueFeedDownloadsAs reads the CALLER's order override and runs episodes through services.OrderEpisodes (same helper as the sequence/reader/page surfaces — the consolidation from item 5a landed with the reverse-order commit). Queue tickets (SEQUENCE-stamped, item 4) make that enqueue order the claim order, so chapters land in the order you'll actually listen. Add modal (§0 item 5c): with 'Download all episodes now' checked, a 'Download order' select appears (pure-CSS :checked sibling reveal — no JS island, CSP-clean). Its choice is the feed's INITIAL order override (reader_settings upsert; 'feed' = no row) AND the queue order for the initial batch. Per-book by design: the override IS the download order. Test TestPodcastAddModalDownloadOrder: add with download_all+reverse → the override row exists AND persisted tickets run position 3→1. Tickets are asserted (not live queued rows) because the shared dev server's worker legitimately races this test's queue — tickets survive every state flip. |
||
|
John O'Keefe
|
0f3c4a8cec |
feat(podcasts): concurrent download pool — fair share, tickets, admin tunables
Replaces the sequential worker with a download-manager-style pool (§0 item 4). Model: the user's connection is the limit; semantics are fair-share ACROSS feeds, not goroutine-per-book (a global FIFO would make Book B wait behind Book A's 100 files). - Atomic claim: GetNextQueuedEpisode + SetPodcastEpisodeDownloading was a read-then-write race under concurrency; ClaimQueuedPodcastEpisode converts queued→downloading in ONE guarded statement (two claimants, one winner — the loser gets ErrNoRows). SetPodcastEpisodeDownloading retired. - Queue tickets: download_queue_pos BIGINT fed by a Postgres SEQUENCE stamped at enqueue — the enqueue loop's ordering IS the claim order (the foundation §0 item 5 builds on; previously all rows in a batch shared one updated_at, so within-feed order was arbitrary). - Pool: up to podcast_download_concurrency (default 4, live range 1–16) claim-loop goroutines per drain round; the setting re-reads every round so admin changes take effect without restart. - Fair share: claim prefers the feed least-recently-served (pickFeedToServe; never-served first), earliest ticket within a feed — books proceed in parallel while chapters land in order (start listening while the rest download). - Bandwidth: shared token bucket across all workers (podcast_download_speed_mbps, default 0 = unlimited); take() reserves bandwidth and callers sleep outside the lock. - Settings: new float setting type end to end (registry defaults + clamping, save validation, admin UI number-with-step input) + the two schema seeds; worker wires the registry via SetSettings (tests run on registered defaults). - Tests: unit coverage for the bucket reservation math + fair-share pick; integration TestPodcastDownloadPool pins ticket order = enqueue order, claim exclusivity, and a full 3-episode pool drain. Dev-DB note: the column/sequence/settings were applied live (idempotent DDL, same statements the schema runs at next boot) since the running container predates them. |
||
|
John O'Keefe
|
e3675536aa |
feat(podcasts): reverse order — position DESC, date-agnostic; single order helper
The 'Oldest first' order sorts by PUB DATE, which breaks on Scribl's migrated podiobooks feeds (messy/all-equal dates). New order value 'reverse' = feed position DESC with a date-DESC tiebreak — dates are not involved, so nothing can break. - services.OrderEpisodes: THE single order source of truth (§0 items 3+5a). The four surfaces (sequence JSON, reader chain context, SSR page list) previously carried three near-identical handwritten comparators — all now route through one generic helper. The SSR page list gains a SortDate key (lexical 2006-01-02) so the pub-date tiebreak is actually chronological; the old comparator compared 'Jan 2, 2006' display strings alphabetically (latent drift bug). - Dropdown becomes four options: Feed order / Feed order (reversed) / Oldest first (pub date) / Newest first (pub date). - Serial suggestion button renamed 'Reverse feed order' (sets reverse, not oldest) and its response out-of-band-swaps the order select (PodcastOrderSelectOOB — same mechanism as Download All's policy select), so the dropdown mirrors the server state immediately. - Validation: oneof=native reverse oldest newest everywhere. - Tests: reverse binds via the form wire; messy-dates sequence returns 3,2,1; existing reorder-safety stays green. |
||
|
John O'Keefe
|
7c7c08748e |
fix(podcasts): download_all binds on every wire — FlexBool + flip log
Reproduced the §0 item 2 suspect: echo v5 bool-binds form values via strconv.ParseBool, which accepts "true"/"1" but REJECTS the browser-native <input type=checkbox> value "on" — and a bind failure kills the WHOLE AddFeed request (400 "invalid request"), not just the checkbox. The add modal posts urlencoded (htmx), so any checkbox-serialization that emits "on" made Download All silently unusable. Fix: FlexBool (encoding.TextUnmarshaler-style UnmarshalParam) binds download_all permissively on every wire — "true"/"on"/"yes"/"checked" => true, explicit false/0/off => false, ABSENCE (unchecked box on any wire) => false; JSON bool binding is native and unaffected. Tolerates the exact wires htmx and browsers emit. AddFeed's subscription flip now logs "[PODCAST] flip ... <feed> — user" so the live flip is provable in app logs (user suspected it never fired). Integration test TestPodcastDownloadAllFormEncoding pins the urlencoded wire (what htmx actually sends): true + on both flip the caller's policy to 'all', an unchecked add leaves the default. |
||
|
John O'Keefe
|
7ea84043fa |
fix(auth): logout actually signs out — clear the HttpOnly session cookie
Release / build-and-push (push) Successful in 3m9s
Logout wiped localStorage but never cleared the token cookie (HttpOnly — only a server response can) and never called the logout API, leaving the refresh token live for 30 days and every cookie-authenticated SSR page accessible: the fake-signout bug. Reproduced on prod and dev; unrelated to RLS (it sits behind authentication) and long-standing. - /api/auth/logout now ALWAYS clears the session cookie (Set-Cookie empty + Max-Age=-1, HttpOnly/Path/SameSite mirroring login) — even a malformed body must sign the browser out; refresh-token revocation unchanged. - The header logout button (and the rules-page copy) now POST to the API with the stored refresh token BEFORE clearing local state and redirecting — best effort: local cleanup never blocks on API failure. - toast 401 handler fixed: it cleared 'refreshToken' but storage uses 'refresh_token' — the real key. - Regression test TestLogoutClearsSession: real login → logout → response deletes the cookie; logged-out refresh token can't mint access tokens; body-less logout still clears. Also carries regenerated artifacts from the confirm-modal and episode-list work (header_templ/podcast_detail_templ/style.css). |
||
|
John O'Keefe
|
30766020c8 |
feat(podcasts): real-time episode list — byte progress bars, direct swaps (issues #1/#3)
WHY: downloads needed manual refreshes to see progress; the 204 + HX-Trigger event indirection missed its listener, and the worker buffered whole enclosures so no progress existed to show. - Worker streams enclosures to disk through a counting loop (64KiB chunks, scheme/redirect/size guards preserved) with an in-memory per-episode progress snapshot (received/total; total from Content-Length, seeded by the feed's declared length). Deliberately transient — the DB row state remains the durable truth. - Episode-list renderer seam: the router injects a closure into PodcastHandler (handlers can't import templates — MediaDetail cycle) so htmx download/retry/Download All/refresh/order/policy actions answer with the REFRESHED partial directly (200, hx-target= #episode-list) — no events to miss. Non-HX callers keep JSON; tests without a renderer keep 204+trigger. - Polling tightens to 2s while downloads are active; downloading rows render a percentage bar + MB counter (quote-free inline style per templ gotcha #8) or an indeterminate bar when the server sends no length. - Issue #3: Download All (button AND the add-feed checkbox) flips the CALLER's subscription policy to 'all' — per-user, never global (D6: files shared, policies not) — and the auto-download select out-of-band-swaps so the dropdown shows what the user is getting. They can turn it down afterwards; the flip is honest communication, not coercion (user-approved 2026-10-29). PINs: Download All with HX answers 200 + episode-list + oob select and flips newest → all; add-feed download_all sets policy all. |
||
|
John O'Keefe
|
c753780817 |
fix(podcasts): order selector validation — bind the htmx form wire format (issue #2)
WHY: htmx sends non-GET requests urlencoded by default; the SetFeedOrder request struct only carried a json tag, so the binder left Order empty and 'required' fired — the exact error the order dropdown AND the Serial-suggestion button produced (hx-vals JSON also arrives form-encoded). One form tag fixes both controls. Regression test drives the exact wire format: urlencoded PUT with order=oldest/native → 200, empty/invalid → 400. |
||
|
John O'Keefe
|
2235b00c59 |
feat(podcasts): web UI — /podcasts page, feed detail, download lifecycle (task 1.9)
Rendering ownership holds throughout: every piece of server data is
templ-rendered; htmx swaps partials; the only TS is the Add-Feed modal
toggle + error surfacing (pure client-local state, bundled via
web/src/podcasts.ts with automatic bundle-hash stamping).
Sidebar: /podcasts nav entry (D13, between Series and Collections),
gated by a ShowPodcastsNav flag resolved centrally in
getTemplateUserWithTheme (one indexed COUNT — same class of
per-request lookup as the live-user guard). New 'podcast' icon.
/podcasts: header + Add Feed modal (htmx post → HX-Redirect to the
feed page; inline error surfacing on failure), subscription-scoped
search + facet bar (host/tag/language — server-side GET filters, no
JS), grid cards with download counts / new-episodes / complete /
explicit badges. Empty states for no-library (admin creates it first)
and no-subscriptions.
Feed detail: hero (artwork, host/publisher, tags, fetch state), action
row — Download All, Refresh, ORDER SELECTOR (D5 override), serial
one-click oldest-first suggestion (never silent), auto-download policy
select (D6), Unsubscribe (files retained), admin Remove Feed — gap
banner (D8 count), and the EPISODE LIST with the full lifecycle inline
(D8/D19): pending greyed + Download button (every non-downloaded
episode has one), queued/downloading pulse, failed RED + reason text +
Retry (same endpoint). The list is a self-refreshing partial: re-fetch
on the podcast:changed HX-Trigger and polls every 4s ONLY while the
server says downloads are active (it renders the polling variant).
API endpoints grew HX awareness: 204 + HX-Trigger podcast:changed (or
HX-Redirect for add/unsubscribe/remove) — the sanctioned Go-handler
response channels; JSON callers keep JSON. AddFeed result exposes the
fetch-error text for the redirect.
Dates render in the requesting user's timezone (D17) via
utils.FormatTimeInTimezone; episodes play through the existing
/readers/{id} audio reader (episodes ARE audiobook items — no new
player code day one). CSP tripwire (templates/csp_source_test.go)
passes: no inline scripts, no on*= handlers, no hx-on/js: vals.
|
||
|
John O'Keefe
|
54d7d6e7a1 |
feat(podcasts): edit segregation — feed metadata is fetcher-owned (task 1.8)
D9/D10: users never edit podcast episode metadata. Three gates: - Generic edit endpoint (UpdateMediaItem) rejects source= 'podcast_episode' items with 400 — the next fetch refresh would clobber any edit anyway, and the fetcher is the sole metadata writer. - Write-to-file (MetadataWriter.WriteMetadataToFile) refuses podcast episodes for the same reason (tag write-back for downloaded episodes is a named Phase-4 item needing its own iTunes-tag mapping). - The book-detail page hides the admin Edit affordance for podcasts-type items (direct-URL visits), and the metadata-editor modal's field list has no podcast fields by construction (fixed templ field list) — episodes render read-only via /podcasts surfaces. |
||
|
John O'Keefe
|
45b1152a15 |
feat(podcasts): API endpoints — add/list/detail, subscriptions, downloads, sequence, search (task 1.7)
The §8 surface, wired through main.go → router.Config → /api/podcasts (JWT). Scoping is the engine's (D20): lookups outside the caller's scope simply return no rows → 404; the only Go-level checks are explicit-argument authorization (AddFeed's library + allow_member_feeds gate) and the admin role on feed deletion. Endpoints: POST/GET /feeds (add: fetch+parse+upsert+subscribe, optional download_all; list: subscriptions + episode counts + facet values computed in Go — feed counts are tens), GET /feeds/:id (detail with download states + gap count), PUT/DELETE /subscription (the ONLY user-writable podcast state, D9; unsubscribe retains files per D6), POST /refresh, POST /episodes/:eid/download (+ episode_ids[] bulk and eid='all' = Download All, exactly the non-downloaded set), PUT /order (D5 per-user override in the reader_settings KV — podcast_order:<feed_id>; 'native' deletes the row so default and explicit choice are indistinguishable), GET /sequence (effective order — override applied, native fallback — DOWNLOADED-only with the gap list; this payload powers web chain mode and, per §15, the future media3 queue), GET /search (subscription-scoped feed + episode search covering un-downloaded and historical rows, D14), DELETE /feeds/:id (admin; best-effort file removal, cascade deletes rows). Gap definition (countGaps): an un-downloaded episode with a downloaded episode AFTER it in list order — the D8 'middle episodes missing' banner condition. |
||
|
John O'Keefe
|
9a148c2344 |
feat(podcasts): generic-surface exclusions — podcasts never render as books (task 1.3)
§9 / D3 / D13 / D15 / D16: podcasts-type libraries are excluded from every generic book surface. Under RLS these are pure UX/type filters (engine already hides other users' episodes); their job is preventing episode grids in surfaces designed for books. The discriminator is the denormalized media_items.library_type_name (IS DISTINCT FROM 'podcasts' — NULL rows predate the column and stay visible). SQL (queries.sql, marked '-- podcasts exclusion (plan §9)'): ListMediaItems/ListMediaItemsSorted (All Books) · SearchMediaItems + SearchMediaItemsUnified (generic search) · FacetCountsUnderFilters (all 14 arms) + the six Search*Values facet-suggestion queries · GetDistinctSeries + Count (series pages) · GetLibraryItems (collections picker) · dashboard system sections (Continue Reading / Recently Added / Recently Read / Not Started) · GetPopularBooks + GetTopBooksByMinutes (book rankings must not rank episodes). Go: statistics summary counts episodes' sessions into time-spent (D15 keeps that) but skips them in BooksActive/BooksFinished — an episode is not a book. OPDS filters podcasts libraries via one helper at all 7 library-walk sites (D16). Templ: LibrarySwitcher skips podcasts entries and TotalMediaCount excludes them — presentation-layer filter so every switcher consumer (dashboard, collections, custom-section builder) is correct at once (D13); podcasts surface via /podcasts only. |
||
|
John O'Keefe
|
c0e6993273 |
feat(podcasts): scanner exclusion — podcasts libraries are feed-managed (task 1.2)
WHY: podcast download roots hold fetcher-written episode files; the FS
walker must never discover them (invariant 4) — a walk would double-
ingest episodes as scanner items and fight the fetcher's sole-writer
rule (D9). Every scan/watch path funnels through MediaScanner.SetFolders
(StartScanner, ScanLibrary jobs incl. scoped single-folder scans,
watch mode), so the filter lives there once: folders resolving to a
podcasts-type library are dropped before registration, with a log line.
Belt and braces: ScanLibrary's per-folder scope refuses purpose='podcast'
folders explicitly (400) — the panel-level path that bypasses type
resolution. AddLibraryFolder now derives purpose from the library type
('podcast' for podcasts libraries, enforced single download root;
'scan' default elsewhere), so folder purpose and library type cannot
diverge through the API.
|
||
|
John O'Keefe
|
d49a9e6ca2 |
fix(devices): device deletion detaches reading history instead of 500ing
Deleting any device whose stats rows still referenced it failed with 500: reading_history.device_id is a NO-ACTION FK by design (the rows are real captured reading stats — Users reads them), so a blind DELETE violated the constraint. Surfaced by the Android app's new logout deregister call, but the admin Devices page Deletes hit the same stone. Fix: DetachReadingHistoryFromDevice — NULL the device association on the user's history rows before the device DELETE (single handler pass). device_id is nullable; the stats survive, the device frees its identifier. sqlc regenerated. |
||
|
John O'Keefe
|
4d697f5fcf |
test(visibility): full suite green under RLS — sync paths, device-auth bridge, seed pool
Phase 5 (plan §7): the entire integration suite passes with the RLS policies live. The suite was the triage enumerator the plan predicted: - device_auth middleware: the THIRD identity bridge — KOReader/Kobo device-authenticated requests read media as the DEVICE OWNER (plan A9 called this path out; the flip was 404s on sync metadata). - conflicts/analytics/progress handlers: detached context.Background() reads threaded to the request context (the requesting user's identity) — 12+ sites; resolve helpers take ctx. - scanner: ScanFolders self-declares its configured admin identity (server maintenance, product-correct for any caller); generic job creation stamps the requesting admin. - harness: setup.DBPool and the sync-test pool declare rls.SeedIdentity at SESSION scope via BeforeAcquire — the pool never serves a second identity, so no A3 bleed; setup.DB falls back to the seed identity (NewSeedDB) for ungoverned-context calls; filter tests use seedExec for raw governed-table statements. - contract pin WIDENED (docs/contract_pins.md 2026-10-29): under RLS, item-scoped gates may answer 404 (row invisible — no existence leak) as well as 403 (Go-level checks on listings keep strict 403). - TestWorker_JobStatusTracking — the pre-existing red from the Phase 0 baseline — passes with the harness seed work; no product change. |
||
|
John O'Keefe
|
bb2cc26e9d |
feat(scanner): scanner entrypoints + generic jobs carry RLS identity; SeedDB harness fallback
The full suite enumerated the remaining triage class: tests drive the scanner directly (ScanFolders(context.Background())) and read seeded media via Queries with bare contexts — every governed read raised. - MediaScanner.ScanFolders self-declares its configured admin (SetAdminID — set by main AND the test helpers): a scan is server maintenance, product-correct regardless of caller. - ScopedDB gains a Fallback identity, exposed as NewSeedDB for the TEST HARNESS ONLY: setup.DB falls back to rls.SeedIdentity when the context carries none, so direct seeding/verification helpers work uniformly instead of per-site context threading. rls.SeedIdentity is the shared constant (no library_visibility rows => full visibility). - generic job creation (jobs.go) stamps the requesting admin into the job Context, same as the scan endpoint. The sync queue processor stays identity-free: sync_queue is not an RLS-governed table. |
||
|
John O'Keefe
|
385a5d9dba |
feat(scanner): background jobs declare acting identity for RLS
Phase 4 of the RLS plan: detached contexts (context.Background()) that reach media_items now carry identities — with the policies live, a bare context made governed reads raise (fresh connections: 42704 unrecognized parameter; reused pool connections: 22P02 empty-string cast after the LOCAL GUC reverted — the plan's A2 signature). - scanner jobs (API-triggered, mtime-poll, initial scan): the job's Context now carries the acting admin's identity — the library owner, first-admin fallback (the job's UserID already encoded this trust decision; the Context just needed to carry it). performInitialScan uses the scanner's configured admin. - websocket initial-sync read (GetUserMediaItemsForSync): runs as the connection's own user. - hash backfill: runs as first admin (server self-heal over all media). Verified on the dev stack: forced rescan of the 1199-file Audiobooks library completed with 0 errors under RLS (previously every SHA lookup and create raised); hash-backfill refusals gone. The sync queue processor needs no identity — sync_queue is not an RLS-governed table. Known remaining detached reads (conflicts/analytics handlers using context.Background()) are HTTP-path triage, enumerated by the full suite next. |
||
|
John O'Keefe
|
9f39e2efd8 |
feat(settings): save feedback — success/error toasts, disabled buttons; fix scan-form raw-JSON swap
Release / build-and-push (push) Successful in 4m35s
None of the three save surfaces on the admin settings page gave usable
feedback: the Base URL form re-rendered into an identical-looking form
(its only indicator was the Go-source twin's banner — markup that
diverged from the templ original), the scan form was WORSE — its handler
responds JSON, so htmx's outerHTML swap replaced the whole scan card
with literal JSON text — and the tunable rows showed only a tiny
per-row label.
Client side (toasts are the sanctioned client-local island; no new
inline scripts, CSP unchanged):
- web/src/settings.ts: one delegated htmx:afterRequest/sendError
listener keyed on a data-settings-save attribute stamped on each
form. Success → green toast per surface ("Settings saved" / "Scan
settings saved" / "Setting saved"); failure → error toast carrying
the server's JSON error field, degrading to a status line for the
tunable endpoint's HTML error body (markup must never reach a toast).
Network-level failures (htmx:sendError) toast instead of silently
re-enabling the button.
- htmx 2.x quirk handled: for FORM submissions, afterRequest's
detail.elt is the event-dispatch ancestor, not the form — the
requesting element lives on detail.requestConfig.elt (verified
empirically; saveLabelFor reads the right one).
- hx-disabled-elt="this" on all save buttons: the click is
acknowledged and double-clicks cannot double-save.
Server side:
- Scan form: hx-swap="none" (the inputs already hold the saved values;
nothing needs re-rendering) — kills the JSON-over-the-card swap.
- The Base URL form's save response is a Go-source copy of the templ
form (handlers cannot import templates: templates/utils.go imports
handlers types). The twin now carries the functional attributes
(data-settings-save, hx-disabled-elt) so feedback survives the swap,
with a drift-warning comment; the double-save e2e pins this.
Dev-DB safety: every success save in the spec re-submits the value
already on the page (zero drift), and the spec restores
system_config.updated_by after itself — the save handler stamps the
requesting admin's id there, and that FK (no cascade) otherwise blocks
the integration harness's transient-user sweep (it broke
TestUpdateProfileAdminMode/TestDeleteUserConsolidated on this run's
first pass; pollution cleaned up by hand and now self-restoring).
Tests: web/src/settings.test.ts pins the form→label map and the
error-extraction contract (including the no-markup-in-toast rule);
e2e/settings-feedback.spec.ts drives all three surfaces through the
real admin UI including the error path and the twin-drift double-save.
|
||
|
John O'Keefe
|
96c1228805 |
feat(series): multi-field search, sort-direction toggle, apply placement
Toolbar feedback round: the series search box now matches whatever the main search matches for a group — series NAME, any book's AUTHOR, and any book's TITLE inside the series (WHERE EXISTS per group-row, so counts stay true); the separate author input is gone from the toolbar (the author= param stays live for the JSON API and deep links). The sort control becomes a base-key select (Date added/Name/Author/Books/ Progress/Last read) plus a chevron direction toggle that flips and resubmits; sort/dir compose through ResolveSeriesSort (full registry keys keep working for back-compat, unknown base keys fall to recent). The search/apply button moved next to the search box as requested, and default URLs stay clean (sort/dir only emitted once the user chooses). |
||
|
John O'Keefe
|
eaf1b2c8dd |
feat(metadata): write-to-file — embedded EPUB OPF + Calibre sidecars
Phase 5a of the metadata-writer feature: the book editor gains a "Write to file" action (admin) that persists the item's CURRENT database metadata into the artifact on disk, so the library stays portable to Calibre and other apps — the thing Save alone can't do. Dispatch (services/metadata_writer.go): EPUB/KEPUB gets its embedded OPF spliced in place (fresh dc:* block generated inside the ORIGINAL metadata open tag; manifest/spine/guide untouched; every other zip entry copied byte-identical in original order so mimetype stays first and STORED; original dc:identifiers preserved — the package's unique-identifier must keep resolving). Everything else takes a Calibre-style sidecar: metadata.opf when the book is ALONE in its directory (Calibre's own convention — round-trips through Calibre too), <basename>.opf when siblings share the folder, so one book's sidecar can never poison dir-mates' scans (the scanner now reads both, basename first). Safety contract: temp file + atomic rename — the original is never truncated on failure; the generated OPF is unit-tested to ROUND-TRIP through the scanner's own parser (write side and read side can never disagree); metadata_overrides are deliberately kept (file and DB agree after the write; overrides still guard against stale sidecars). The endpoint re-extracts inline (RescanMediaItem) and reports rescan failure separately — a failed rescan doesn't undo the write. SHA-256 changes are inherent and documented for users (Kobo/KOReader re-key). UI: the metadata editor modal carries the file path as a data attribute; the button confirms before mutating, labels the target (embedded vs sidecar) and toasts the outcome. Integration tests pin the 403, missing-file error, EPUB embed + post-write rescan title, the lone-book metadata.opf case, and the shared-dir <basename>.opf case with no dir-scoped litter. |
||
|
John O'Keefe
|
61e5f47ec0 |
feat(search): viewer progress + completed badges; dropdown to templ
Search results now show where you are in a book: a completed check at the 0.999 threshold (same cut as the filter status arms), a partial progress bar along the cover while mid-read, nothing when you never opened it. Rendered in BookCard (any grid whose adapter loads viewer progress) and in the header-search dropdown rows. The progress rides a LEFT JOIN reading_progress on the SAME user_id param SearchMediaItemsUnified already binds for library visibility — one requester param, no new wire inputs, row-neutral join (the table is UNIQUE per media_item+user). SearchMediaItemsUnifiedRow and BookInfo gain the keys additively (viewer_percentage null when unknown, library_name empty on feeds that don't load it); contract_pins.md carries the changelog entry and the pin tests widen in lockstep. The header dropdown itself migrates off the innerHTML string renderer in search.ts (a rendering-ownership violation: a templ twin existed via the HX-Request branch). The input now hx-gets layout=dropdown and swaps the new templ SearchResults partial — titles/authors highlighted server-side (escape-first, safe by construction), rows with the same progress overlays — while a slim Alpine component owns only client-local state (min-length gating, open/close, keyboard selection, htmx lifecycle). web/src/rendering-allowlist.txt drops the search.ts baseline to zero accordingly. Integration test pins the per-user join (fresh user sees 1.0/0.42/null on the fixture; the shared admin sees null everywhere — read-only) and the dropdown partial's overlays/count/row count. |
||
|
John O'Keefe
|
d1f9ad3b25 |
feat(series): sort and filter the series page
The /series page rendered its grid in one fixed order (newest first)
with no way to find a series beyond eyeballing cards. The page now has
a toolbar — series-name search, author filter, viewer-progress filter
and a 12-key sort select — wired as a plain GET form (the page's
existing model: the URL is the single source of truth, and the library
switcher now carries q/author/status/sort along on switch).
Extensibility is deliberate (user request): sorts live in a registry
(services.ResolveSeriesSort) mirrored by a CASE cascade in
GetDistinctSeries and a <select> in series.templ — a new sort is one
entry in each. The sort value is a BOUND PARAM compared inside the
cascade, never interpolated; unknown keys resolve to the default.
Filters: series-name matches in WHERE (the group key, so counts stay
true); author (BOOL_OR any-book match) and status (finished/reading/
unstarted vs the viewer's progress, same 0.999 threshold as the
filter_query status arms) in HAVING so COUNT(*) keeps reporting the
whole series. GetDistinctSeriesCount shares the JOIN + HAVING so
pagination totals match. The reading_progress join can't multiply rows
(the table is UNIQUE per media_item+user).
Cards gain the viewer's finished-count ("X of N read" + progress bar),
and the empty state distinguishes "no series at all" from "no matches —
clear filters". The JSON /api/series endpoint takes the same params and
gains read_count/last_read_at additively.
Integration test pins every sort key against a three-series fixture
with distinct created/author/count/progress answers, the filter matrix,
composition, unknown-sort fallback, card read-counts and the JSON
contract — all as the fresh regular user so the live reader's data is
untouched.
|
||
|
John O'Keefe
|
c71ef4af85 |
fix(auth): show error feedback on failed htmx login/registration
Found while double-checking /login after the redirect fix: a wrong password left the form completely silent. Two stacked causes, both pre-dating the CSP work: 1. The two HX-Request branches for invalid credentials returned JSON, unlike every sibling error branch — there was no HTML partial to show at all. They now return a shared invalidCredentialsHTML div (the JSON shape is kept for non-htmx API callers). 2. htmx does not swap 4xx/5xx responses by default, so even the HTML error partials that DID exist (429 lockout, 500s, registration failures like "Email already exists") never landed in the target. auth_forms.ts now swaps them itself on htmx:beforeSwap when the error response is text/html — JSON errors stay ignored. Verified live: /login success → /dashboard; wrong password shows "Invalid email/username or password." and stays on /login; duplicate email on /register shows "Email already exists". Gates: make test 9 pkgs, vitest 155, Playwright 20. |
||
|
John O'Keefe
|
7c9e561e68 |
fix(auth): login/registration forms redirect again under hash CSP
The htmx login/registration partials answered success with an inline <script> that stashed the auth token and set window.location — refused by hash-mode script-src (no 'unsafe-inline'), so the user saw "Login successful! Redirecting..." and nothing else (reported from the home- page login form; /login was equally broken). The same pattern lived in collections.go (delete → redirect) and devices.go (token regen → reload, whose script tag was also unclosed). Replacement, per the AGENTS.md CSP rules: - auth.go login/registration: the partial now carries data-auth-token + data-redirect (redirect is path-only — open-redirect hygiene — and escaped); web/src/auth_forms.ts (new, armed from main.ts on every page) listens for htmx:afterSwap, stashes the token, and navigates. - collections.go delete: data-redirect, same listener. - devices.go token regen: htmx's native HX-Refresh response header. - The dead localStorage 'user'/'refreshToken' writes are dropped — no reader anywhere in web/src; only 'token' is load-bearing (19 modules). Tripwire extension: internal/middleware/csp_go_sources_test.go scans non-test Go sources for literal <script> tags and native on*= attributes (hand-written files only — templ codegen is owned by the .templ scan), verified to fire on a reintroduced probe. AGENTS.md CSP section notes the Go-emitted-partial vector. Verified live: home-page login form → /dashboard with token stashed and dashboard loading clean; /register form → /dashboard; collections and devices share the same mechanisms (data-redirect / HX-Refresh). Gates: make test (9 pkgs incl. both tripwires), vitest 155, Playwright 20, tsc/vp clean. |
||
|
John O'Keefe
|
0f1d0abbb0 |
feat(libraries): scope a library scan to one registered folder
POST /api/libraries/:id/scan accepts an optional {"folder_path": "..."}:
the scan then covers exactly that one registered folder's subtree instead
of the whole library. The path is validated against the library's
registered folders — anything else is a 400, never trusted. An omitted
body keeps today's full-library behavior, and coalescing (409 with the
active job) is unchanged.
The LibraryPanel folder rows get a per-folder Scan button: pure htmx
attributes posting to the same scan endpoint, with hxValsFolderScan
emitting plain eval-free JSON for hx-vals alongside the existing
hxValsFolderDelete precedent. Progress surfaces through the existing
active-scan polling; no new client code.
Safety: the scanner's archive-reconciliation guard (previous commit)
skips partially-covered libraries, so a one-folder scan cannot mark the
library's other folders' items missing.
|
||
|
John O'Keefe
|
6f37be59fc |
feat(issues): retry a processing issue through the individual scanner
POST /api/libraries/:id/issues/:issueId/retry (admin, family-lens gated via the existing gateIssue) re-runs the work that produced an open issue — deliberately NOT a library-wide rescan. Two shapes mirror the two ledger entry kinds: - item-backed issues rescan their media item through the book-detail RescanMediaItem mechanism (self-resolving folders, multi-file audiobook routing, write-after-success guards) and then close the issue (RetryMediaItemIssue); - row-less import_failed issues re-import the single file through the watcher's standalone processMediaFile path (RetryFailedImport: folders resolved RescanMediaItem-style so paths stay library-root-relative), whose create-path self-heal plus an explicit path resolve closes the issue even when the content was already imported by a prior attempt. Failures return the REAL underlying error so the UI can show why the issue still exists; a vanished file is a 409 (ErrRetryFileMissing); already-resolved issues reject re-retry. Docs + bruno mirrors regenerated for the new route (the generator's seq renumbering touches three unrelated bruno files as fallout; openapi gains the path and the date bump), and the docs schema registry gains the retry entry so TestDocsMatchRoutes passes. |
||
|
John O'Keefe
|
c645b8e207 |
feat(profile): keep_screen_awake user preference (default on)
New users column surfaced as an On/Off dropdown in the profile form. The form and API carry "true"/"false" strings with the theme-style ""-guard, so JSON clients that omit the field can't clobber it; anything else is rejected. GET/PUT /api/auth/profile expose keep_screen_awake alongside the other profile fields, and the reader route threads the preference into its template metadata. Integration tests pin the schema default, both values, omission semantics, and rejection — dedicated test user, never testuser. |
||
|
John O'Keefe
|
bdb9132b85 |
chore(auth): drop dead UpdateProfileRequest; rename shared struct
The combined profile handler (self-edit PUT /api/auth/profile and admin PUT /api/auth/profile/:id) binds one struct for both paths — and it's the admin-named one. Its twin UpdateProfileRequest had zero references left. Delete it and rename AdminUpdateUserRequest to ProfileUpdateRequest so the name matches who actually binds it; the comments describing field semantics now stand on their own instead of pointing at the deleted twin. Wire format (json/form tags) unchanged; docs schema registration follows the rename. |
||
|
John O'Keefe
|
d41c3e4774 |
fix(ui): hide Comic/Manga editor section for non-comics media
The metadata editor's Comic/Manga section (Manga Type, Reading Direction, Age Rating, Story Arc, Imprint, Scan Info, Metadata Notes, B&W) rendered for every item — an audiobook showed a manga dropdown, which is nonsense. MediaDetail now carries LibraryTypeName (resolved via GetLibraryType in the /media/:uuid handler) and a ShowComicMetadata() gate: comics and manga libraries always show the section; other media only when the scanner recorded comic-style values (fixed-layout manga-style EPUBs get those scanner defaults deliberately — when present they stay editable). |
||
|
John O'Keefe
|
a716b386a1 |
fix: wire security headers, graceful shutdown, and settings invalidation
Lifecycle (internal/app + main.go): - App.OnShutdown hooks run after the HTTP server drains, in registration order: Worker.Shutdown (background jobs now drain instead of dying on SIGTERM), DeviceAuthMiddleware.Stop (releases the rate-limiter ticker), Handler.CloseScanner (fsnotify watcher and scan log files) Security headers: - SecurityHeadersMiddleware (survivor of middleware/security.go, renamed security_headers.go) is now on the echo stack: nosniff, frame-deny, HSTS, CSP (unsafe-inline required by the sanctioned theme flash guard + Alpine), referrer and permissions policy Scanner settings cache: - settings saves (UpdateScanSettings + ApplySetting for the three scan keys) now invalidate MediaScanner's 30s settings cache via SystemSettingsHandler.SetScanSettingsChanged, so changes apply on the next poll instead of after the TTL - MediaScanner.Close closes the scan logger, fixing a file-descriptor leak on every per-item rescan (fresh scanners never closed logs) Client fix: - custom-section-builder's add/remove-selection buttons used inline onclick attributes referencing module-scoped functions, which are not global in a bundled ES module: the buttons threw ReferenceError. Functions restored and exposed on window via declare global Also fixes a pre-existing vet error (saveCoverImage copied echo.Context by value, mutex included) and drops the per-key SettingsCache.InvalidateKey variant left callerless by the full-flush invalidation wiring. |
||
|
John O'Keefe
|
8631aa00bf |
refactor: remove dead code across server, templ pages, and client modules
Deadcode-verified removals (RTA reachability from main, cross-checked with -test and textual xrefs): Go server: - whole files: middleware/transaction.go, middleware/error_handler.go, middleware/password_validator.go (live registration machinery kept), handlers/sync.go, sync/offline.go - unrouted endpoints: auth UpdateUsername/UpdateEmail, timezone settings handler, OPDS RegisterOPDS, Kobo LibrarySync handler, ReaderHandler.ShowReader, JSON-data twins on collections/library - duplicate book-matching handlers (book_matching.go Handler twins, matching.go MatchingHandler twins), superseded helpers (router.parseUUID, handlers.ValidateDeviceToken, normalizePath, userExists, generateLockID) Templ/client: - never-routed pages: api_explorer.templ, queue.templ and their client modules api-explorer.ts, api-explorer-docs.ts, queue.ts - fully orphaned wrapper modules events.ts, dom.ts - broken clearSyncQueue (called a nonexistent endpoint) and the hidden, never-populated sync-queue section on the devices page - 166+ unused exports demoted to private (knip-verified); ~85 truly dead declarations removed via tsc noUnusedLocals fixpoint - stale 'interactive API Explorer' claims dropped from docs The docs drift-gate package (internal/docs), templ-bound modules (dashboard, stats, header, index, login, profile*, bookshelf, collections, toast-error), sqlc-generated code, and web/static are intentionally untouched. |
||
|
John O'Keefe
|
8733e83f1f |
style: apply gofmt across Go sources
Repo-wide gofmt sweep: const-block alignment, doc-comment spacing, statement-after-brace fix in media.go, trailing blank lines. No semantic changes — whitespace-insensitive diff is empty for every file in this commit. |
||
|
John O'Keefe
|
c4b62af213 |
docs(devices): Work Order 3 batch 1 — 6 missing device endpoints + gate entries
New frontmatter'd docs with handler-compatible examples: - devices/get_device.md (DeviceInfo shape) - devices/regenerate_device_token.md (token + sync_urls) - devices/get_device_mappings.md (wrapper + DeviceMappingSummary) - devices/update_device_mapping.md (UpdateDeviceMappingRequest) - devices/download_sidecar_config.md (sidecar config JSON file) - devices/link_book.md (LinkBookRequest; one UNVERIFIED marker on required-tag) Schema registry: 6 new entries; DeviceMappingSummary hoisted to package scope. Gate passes (the UNVERIFIED marker is logged, not a failure). Excludes /api/devices-page (htmx page route, per ground rules). |
||
|
John O'Keefe
|
cc893f758e |
feat(docs): drift gate — docs frontmatter validated against real routes
Every API doc that declares frontmatter (method/path/auth/area) is now checked by a unit test against three sources of truth: 1. Route registration: internal/router/*.go is parsed the way echo resolves it (group prefixes concatenate; scanner.go's parameterized groups seeded). Doc paths that no longer exist fail. (This is the check that would have caught the /api/api/admin/hash-conflicts split.) 2. Auth: frontmatter auth label must match the middleware chain (none|jwt|jwt+admin|device-token|optional-jwt). 3. Shapes: internal/docs/schema.go registry — example-request decoded with unknown fields DISALLOWED (typo guard), required-by-validate fields must appear (staleness guard), example-response top-level key set compared both directions against the marshaled handler type (or a RawMessage pin for hand-built map responses). Five collections docs are the pilots with real handler shapes; notable corrections they pin: update returns 201 (not 200), list excludes smart collections unless include_auto=true, rule priority is 1-10 higher-wins (docs claimed 1=highest), created_at is Go time.String() not RFC3339, hide_finished/show_up_next toggles documented. CollectionResponse hoisted out of GetCollections/GetBookCollections as CollectionSummary / BookCollectionSummary so the registry can pin real types. The gate fails on missing registry entries by design — that is the maintenance contract. Known pre-existing (not this commit): vet copy-lock warnings in media.go. |
||
|
John O'Keefe
|
3576a4415a |
feat(audio-player): per-book playback speed + user default (plan §8)
Speed becomes a per-USER-per-BOOK preference that syncs across devices, with a user-level default for books not yet played: - new audio_playback_speed table (mirrors reading_speed's shape; a preference, NOT a measurement and NOT a reading_progress column) + users.default_audio_speed, schema in place - chapters endpoint resolves per-book row → user default → 1.0 and ships playback_speed + default_playback_speed (zero extra round trips; default powers the dropdown marker) - PUT /api/media-items/:id/audio-speed upserts (range-checked; the client SPEEDS list stays the real gate; library visibility gates writes) - the user default rides the profile path like theme/timezone: profile form gains a 'Default Playback Speed' select (server-side SpeedOptions mirrors the web SPEEDS list), GET /profile carries it - web: restorePosition applies the resolved server speed (localStorage audio_speed demotes to a first-paint cache); setSpeed PUTs fire-and-forget with a failure toast; the Speed dropdown gains a '(default)' row marker and a 'Make N× my default' action with a success toast ALSO fixes a pre-existing profile-page bug surfaced by the §8 form verification: the password section's current_password input was required INSIDE the shared form, so HTML5 validation blocked every Save Changes submission (dead since the shell migration). The password flow keeps its own gates (disabled-button checkers + server 400). Integration test TestAudioPlaybackSpeed pins the full contract (unset → 1x, upsert resolution, range rejection, default fallback for row-less books, default updates re-satisfying unset books). Verified live: 9 Playwright checks. |
||
|
John O'Keefe
|
35d8ba32e5 |
fix(media): metadata editor save no longer wipes audio_metadata
PUT /api/media-items/:id (the metadata editor) writes the FULL row via UpdateMediaItem, whose query sets audio_metadata unconditionally — and audio_metadata has no editor field, so the params literal left it nil and every save NULLed the payload. A plain metadata edit (genre tweak, title rename) dropped any audiobook back to "1 second long" with no track manifest: the same incident class as the scanner wipe, sitting on the most-used admin path in the app. Carry existing.AudioMetadata through the write. Write-after-success: only the audio chain (probe-backed extraction / UpdateMediaItemAudioBookPayload) ever replaces that column. |
||
|
John O'Keefe
|
9b5defcb31 |
feat(issues): the honest processing-issues ledger — row-less import failures, self-healing resolution, family lens (faceted plan Phase 11)
processing_issues only recorded cross-library move rejections; files that failed to import vanished into an error counter and stdout. The ledger now records every failure class and heals itself: Schema (in place, pre-release boundary): - media_item_id NULLABLE + file_path TEXT (deliberately unbounded — an over-long path is itself a failure mode; a VARCHAR(500) ledger column died on the exact 22001 it existed to report — caught by the new test). Row-less failures key on (library, file_path); item-backed on (item, type). Widened dedup = COALESCE-expression unique index; the old table UNIQUE stays (NULLs pairwise-distinct for row-less). - Conflict-target upsert: repeat failures UPDATE the same row and REOPEN resolved ones — never duplicates. Wirings: - CreateMediaItem failure (single-file + multifile createAudioBookItem) → row-less import_failed (error). - Extraction hard-fail with fallback import → item-backed metadata_extraction_failed (warning). - Change-path re-extraction failure (the ev/sidecar guard) → item-backed metadata_extraction_failed; LogProcessingIssue keeps its signature, a row-less helper joins it. Auto-resolution: - Successful import → ResolveOpenProcessingIssuesForPath (library, file_path). - Successful (re)extraction → ResolveOpenProcessingIssuesForItem (single-file change path, multifile editorial apply). - Unchanged-skip scans do NOT resolve: extraction never re-ran, so the degraded metadata is still degraded — the page heals exactly when the book does. Family lens on ALL issues surfaces (list, stats, resolve, delete, SSR page): explicit out-of-set library id → 403, never silent. This tightens one pre-lens behavior — a nonexistent library id returned 200-empty, now 403 (pinned, changelog entry in docs/contract_pins.md). List is LEFT-JOINed so row-less rows appear (media_item_id "", title falls back to the path). Tests: full lifecycle state machine through real scans (fail → appears, repeat → no dups, success-at-path → resolved, LEFT-JOIN list shape); family lens 403s with a fresh non-shared admin; all 7 pre-existing issue suites green (two expectations widened to the lens, changelogged); adjacent suites (CrossLibraryMove, Audiobook, Calibre, ExtractorVersion, SchemaIdempotency) green; unit suites green. |
||
|
John O'Keefe
|
9c006a9a2f |
fix(filters): browse with no rules works, popup under the input, no sticky headers (user report)
- Blank typed queries rode the STATIC autocomplete path for the classic six fields, where a single space similarity-matches into nothing — clicking an empty value box with no other filters said 'No matching values' in a library full of values. Blank = browse now always rides the composed engine (which trims and answers the top values); static keeps its pinned contract for real typed queries. Regression-pinned: browse with no rules lists the field's values. - The value popup now drops BELOW the input (top: 100%), not over it. - Field-menu group headers lost position:sticky — the search box alone stays pinned. |
||
|
John O'Keefe
|
4ee24e7152 |
feat(filters): the rule builder — composed engines over the field registry (faceted plan Phase 9)
- Field registry (services/filter_registry.go): ~55 typed fields — every book column, file/lifecycle facts, the three HOLDING STRUCTURES (library, collections with excluded-aware membership, series with FULL primary+alternate_info membership), and the whole user-relative class (status, progress, last read, my rating, review, notes/highlights presence, note/highlight text, time spent, pages read, collections, added by) reading the VIEWER's rows. Operators are typed constants; negation is an operator. - Composed engines (services/filter_query.go): one Go clause builder over raw pgx feeds the search (identical SearchMediaItemsUnifiedRow shape), the field-aware suggestions, and a true-COUNT live count. Mandatory WHERE head everywhere (family lens + integrity facts). Values are always bound parameters — no user string reaches a field/operator position, ever. - Wire: the qf JSON param (validated strictly — unknown fields/operators/malformed values are 400); qf wins over flat params on hand-crafted URLs; flat deep links keep working unchanged. - Endpoints: GET /api/media-items/filter-count (live count); field-values autocomplete upgraded in place — any registry field key, qf binds the OTHER rules (symmetric under-filter suggestions, zero-match values never offered); classic six without a tree keep their static pinned contract. - UI: the drawer becomes the rule builder — the ONLY filter UI (templ markup, Alpine tree state, sanctioned island; grouped field menus, operator menus with red negations, field-aware value menus, live count, chips per rule with red negations, URL rehydration, smart serialization flat-when-simple else qf). - Tests: composed semantics (AND/OR/negation/nesting/enums/bools/ numbers/dates/empty), every user-relative field class with the my-data weld (two readers, different worlds), note-text substring pin (catharsis ≠ Carthage), family lens (hidden libraries nowhere, 403s), suggestion-under-rules exactness, qf validation + precedence, flat-vs-tree equivalence; registry completeness unit test. |
||
|
John O'Keefe
|
ebb7f4a5b0 |
feat(filters): panel removal + All-Books field-values fix (faceted plan Phase 8)
- Remove the Phase 7 facet-panel UI: SSR route, facet_panel.templ, FacetCountsForView, drawer panel block, panel TS + .facet-* CSS. The ENGINE stays (FacetCountsUnderFilters, all-facets endpoint, drift guard) — it becomes the Phase 9 builder's machinery. - Fix the All-Books bug: field-values autocomplete (author/genre/ tags/narrator/series/language) treats library_id as OPTIONAL — absent aggregates every VISIBLE library (family lens in-query); explicit out-of-set library_id is now 403, never a silent pass. The four scalar Search*Values queries get the nullable shape. - toast.ts: suggestion fetches are exempt from error toasts (console-only) — a failed background hint never interrupts. - TestFacetPanelPartial deleted with the behavior; TestFieldValuesAll- Libraries pins the fix (200 + cross-library aggregation, family lens, 403). Contract-pins changelog entry added. |
||
|
John O'Keefe
|
f52bb6f073 |
feat(facets): all-facets panel with under-filter counts (faceted plan Phase 7)
The 9 Batch A facets could not be SET from the web UI, the facets
endpoint had no in-app consumer, and the registry's libraryTypes hints
rendered nowhere. This phase closes all three.
- ONE UNION-ALL sqlc query (FacetCountsUnderFilters) computes all 14
listable facets in a single round trip: each arm omits its OWN
facet's filter block and carries every other registered facet's
block (copied mechanically from SearchMediaItemsUnified) — classic
faceted semantics: with filters active each facet counts under the
others; with none, blocks are neutral and the counts are the v1
global-per-library ones. One query serves both cases. The 14
per-column Batch C queries are deleted.
- Endpoint: no facet param → the new all-facets shape
({facets: {param: [{value, display?, count}]}}), filters bound via
the shared ParseMediaSearchParams, library_id optional (the
All-Libraries view, family lens still enforced); facet=X keeps the
pinned single-facet shape (now routed through the same query with
neutral filters).
- SSR partial GET /media-items/facet-panel: templ renders the drawer's
value lists — registry order, Batch D library-type scoping
(FacetsForLibraryType: manga_type on manga, reading_direction on
manga+comics, narrator on audiobooks; All view = union), prettified
genre labels + language display names, active values marked. Family
lens: out-of-set explicit library_id is 403.
- Drawer wiring: the panel fetches on drawer open and after every
apply with the CURRENT filter query string; a value click toggles
that facet's comma-joined filter (hidden input where the drawer has
no field — same as deep links) and re-applies the form; chips and
URL stay in sync. Values are server-rendered (rendering doctrine);
bookshelf.ts owns only the click/fetch orchestration.
- prettifyGenre moves to internal/utils as the canonical
implementation (services' display enrichment shares it; templ wraps
it; the TS port in bookshelf.ts is unchanged).
- Drift guard extended: FacetCountsUnderFiltersParams must carry every
registry facet's param, the bools as pgtype.Bool.
Tests: under-filter exactness (own-block omission, OR-within,
AND-across), family lens (hidden values absent; 403 on both shapes),
all-shape + pinned-shape coexistence, partial rendering with Batch D
scoping + active marking + 403. Full container suite green except the
documented pre-existing TestAdminCreateUser red.
|
||
|
John O'Keefe
|
5245f292ca |
feat(scanner): quiet scans, enqueue coalescing, watcher probe, sibling cache
Part 2 of the faceted plan, Phase 5 (steps 5-6): the scan that used to print ~4 lines per unchanged file now prints ONE summary line, and a poll tick landing mid-scan no longer stacks a second full scan. - Quiet scans (§4): the walk loop is silent — inserts, changes, and errors log inside processMediaFile; the duplicate 'Successfully processed' branch (both branches printed it) and the per-file 'processing / already exists / same size, skipping' chatter are gone. Summary carries the trigger and a changed counter: 'Scan of <folders> (trigger: poll): N files, N changed, N new, Xs, N errors'. Triggers threaded through every enqueue site: poll (startBackupScan), watcher (dirty-dirs flush), admin (scan API), initial (startup scan). - Enqueue coalescing (§5): Worker.HasActiveScanForLibrary (JobResult gains additive job_type so only SCAN jobs coalesce — an import sharing the library id never blocks one). enqueueLibraryScan skips silently; the admin scan API answers 409 with the active job id. Enforces the one-scan-per-library invariant the bulk scan-state map leans on. - Watcher capability probe (§4): ONE Add on a root folder; failure disables the watcher with a single startup line (replaces the per-directory warning spam on inotify-broken deployments). The watcher stays an optional latency reducer over scheduled scans. - Per-directory sibling cache (§6): hasSiblingBookFileCached caches the ReadDir per directory for one scan — loose-image manga re-stats thousands of images sharing a handful of directories. Tests: TestHasActiveScanForLibrary (unit — running/pending coalesce, terminal jobs don't, non-scan jobs don't, empty library never, nil worker safe). Full scanner suite green in the container. |
||
|
John O'Keefe
|
7e586df48a |
feat(facets): facet registry, Batch A server params, genre model v2, facets endpoint
Phase 1 + backend of Phase 2 of the faceted-navigation plan. One
declarative facet registry (services/facet_registry.go) drives the
bookshelf filter system; a drift-guard unit test reflects over the
generated SearchMediaItemsUnifiedParams so a registry entry and its
sqlc.narg block can never disagree silently.
- Nine new filter params on SearchMediaItemsUnified: story_arc,
imprint, edition (scalar-fuzzy), age_rating, format_group,
manga_type, reading_direction (exact), explicit, abridged
(presence-bound bools). Comma = OR-within multi-select on every
text facet; AND-across-facets unchanged.
- Genre model v2 (user rule: a subgenre is not a genre, it's a tag):
genre_lexicon.go rebuilt as shelf set + subgenre->shelf map;
pickSmartGenre picks direct shelf > mapped subgenre > Fiction floor
> empty (unknown-only stays tags-only). Type-words (Manga, Comics,
Graphic Novel) join the demote list. Both 'adventure' and 'action &
adventure' are shelves — provider data uses both and the verified
traces pin them.
- Language filter UX: comma elements normalize through the ingest
language map, so display names ('English') match rows stored as ISO
codes ('en'). The language autocomplete now serves display names.
- NEW GET /api/media-items/facets?facet=X&library_id=... — registry-
driven value+count listing (14 static per-column queries, family-lens
join); language values carry display names; bool/range facets 400.
- Family-lens gates: an explicit hidden-library library_id is now a
403 on the search endpoints (was a silent empty set) — JSON branch
and HTMX grid partial both.
- handlers.ParseMediaSearchParams is the SINGLE param parser for the
search JSON endpoint and the HTMX grid partial; the HTML branch
previously dropped publisher_filter and narrator_filter silently.
Pins: docs/contract_pins.md changelog entries (additive request
params; new facets endpoint). Tests: facet_registry_test.go (shape +
SQL drift guard + library-type union), series_parsing_test.go genre-v2
cases, facets_integration_test.go (param filtering e2e, counts,
language display, 403s) — container runner.
|
||
|
John O'Keefe
|
7537063cb7 |
feat(filters+nav): author navigation end-to-end — arrays on every media response, deep-linkable bookshelf filters, clickable authors
Completes the A2c acceptance scenario: click an author on a book detail page -> land on the bookshelf filtered to everything that author wrote (including their share of co-authored books). - Media detail + unscoped list resolved maps expose subtitle, explicit, authors, publishers (the library-scoped raw branch and the search endpoint already did — raw rows follow the SELECT). - SearchMediaItemsUnified SELECT carries the new columns; the search endpoint serializes the row directly, so clients get them there too. - book_detail.templ renders each author as a link to /bookshelf?author_filter=<name>. - Bookshelf honors filter params in the URL on load (author_filter, series_filter, genre_filter, publisher_filter, tags_filter, narrator_filter, language_filter): pre-fills the form, applies it to the first grid load. Comma-joined values are multi-select via the server-side string_to_array split. Filtered views are shareable. Pins widened: search rows (+7 keys), resolved keys (+4); changelog entries in docs/contract_pins.md. Android follow-up plan written to ~/.opencode/plan/bookhoard-app-v2-compat.md. |
||
|
John O'Keefe
|
f94e7472b4 |
feat(filters+ui): per-element author/publisher filters, playlist toggles, Up Next API, reorder UI, dashboard badges
Phases 4 + 5 of the metadata overhaul (plan §4, §5) plus §3 handler wiring. Filters (§4.1): author_filter/publisher_filter now match per-element against the authors[]/publishers[] arrays — a co-authored book matches any one of its authors — with comma-joined params acting as OR-within-facet multi-select (string_to_array + cross unnest). Legacy single-value params unchanged on the wire; series/genre/language filters stay on their varchar columns; narrator filter already per-element. Collections API (§3.3-3.5): collection detail carries viewer + hide_finished into the listing query and exposes up_next (visible- library-enforced reading-queue head); PUT /api/collections/:id/items/order (owner-only, full-order rewrite); UpdateCollectionRequest gains hide_finished/show_up_next ([] string last-value-wins for the hidden-false-sibling checkbox pattern) merged server-side OVER stored view_settings so imported_from_series provenance survives edits; EditCollectionModal renders the toggles from view_settings. Dashboard (§5.5): sections JSON carries is_playlist/complete (dashboard.ts is the AGENTS-documented JSON-render exception; first paint stays templ). Fully-read playlists render '✓ Playlist complete' instead of an empty rail; imported playlists get a 📖 Playlist badge. Frontend: book detail shows subtitle + full authors + Explicit badge; edit modal gains subtitle, comma-separated authors/publishers (client- split to arrays exactly like narrators), step=any series number, explicit checkbox; collection book cards get playlist move ↑/↓ buttons persisting the full order via the reorder endpoint (templ partial re-render on failure — DOM order is client-local, API order is truth). AGENTS.md gotcha #3 amended per plan §3.7: user collections are dashboard sections by default; the show_on_dashboard premise is retired. Verified: full affected integration sweep green (pins widened for up_next/is_playlist/complete — changelog entries in docs/contract_pins.md); unit suite green; go build + templ + vite green. |
||
|
John O'Keefe
|
1dbb0ce406 |
feat(collections): playlist auto-creation, position ordering, hide_finished, Up Next, dashboard eligibility
Phase 2b + 3 of the metadata overhaul (plan §2.4, §3.1-3.7). Auto-playlist collections (§2.4): the scanner turns obviously-playlist sidecar series (explicit '(Reading Order)'/chronological/publication/ in-order/timeline markers only — precision over recall) into curated static collections: upsert by view_settings.imported_from_series then name (rename-safe), position set ONLY on insert (clean integers; 1.5/1a stay unpositioned rather than guessing), membership add-if-missing, never removed or re-ordered by rescans, provenance in description + view_settings, show_on_dashboard true (dashboard-eligible by design), 'scanner.auto_playlist_collections' setting (default on, Settings Registry + schema seed). Ordering (§3.1-3.2): collection listings order position ASC NULLS LAST then added_at ASC (was added_at DESC — walked playlists backwards); PUT /api/collections/:id/items/order applies full-order rewrites (owner-only, service-gated). hide_finished + Up Next (§3.3-3.4): GetCollectionItemsForDashboard gains viewer_user_id + hide_finished (LEFT JOIN reading_progress, COALESCE percentage < 0.999) and viewer_percentage rides the row for card graying; GetCollectionUpNext serves the collection's reading-queue head with the family-lens visible-library array enforced. Dashboard (§3.5): user collections are dashboard-ELIGIBLE BY DEFAULT — the show_on_dashboard gate (never writable for user rows since the column shipped) is gone from GetUserCollectionsForDashboard; hiding stays dashboard-prefs (hidden_collections) territory; hide_finished filtered-everything now emits a Complete section instead of silently vanishing; sections JSON carries is_playlist/complete. Series page ordering hardening (§3.6): GetSeriesBooks tiebreaks by title ASC — NULL/tied numbers previously returned heap (scan) order, which is what the user saw. Frontend custom-section-builder note: the builder has sent show_on_dashboard: true since its creation; the API never bound it. User-row visibility is now intentionally prefs-governed, so the field stays unbound by design (documented in code). Pins widened (docs/contract_pins.md): collection detail + up_next; dashboard sections + is_playlist/complete; new reorder endpoint recorded. Verified: TestPlaylistCollectionsEndToEnd (new, full e2e: sidecar series split, collection auto-creation with position 7, rescan idempotency, owner-only reorder, finished-filter + completion marker + Up Next drain) plus the full affected integration sweep green. |
||
|
John O'Keefe
|
646477afad |
feat(schema): subtitle/explicit columns, authors+publishers arrays, float series numbers
Phase 1 of the series/collections/metadata overhaul
(~/.opencode/plan/series-collections-metadata-overhaul.md).
Schema (in place, idempotent):
- media_items.series_number INTEGER -> DOUBLE PRECISION: real-world
ordinals are decimals (0.5, 1.5 novellas); INTEGER silently truncated
them (calibre 2.5 pinned as 2 in tests).
- + subtitle TEXT, explicit BOOLEAN (sidecar ingest; ABS explicit /
Audible isAdult).
- + authors/authors_search/publishers/publishers_search TEXT[] NOT NULL
(narrators A2a precedent). Arrays are write truth; author/publisher
varchars become trigger-regenerated display strings
(sync_author_publisher_display) so every varchar consumer (trgm
search, facets, API, OPDS/Kobo) keeps working unchanged.
- Row-level NULL normalization for the new arrays in
set_library_type_name (A2 lesson: Go nil -> explicit NULL -> 23502).
- Backfills ARRAY[author]/ARRAY[publisher] — deliberately no
comma-splitting ('Tolkien, J.R.R.' would corrupt; scanner kept only
authors[0] historically).
- collection_items.position INTEGER + (collection_id, position) index
for playlist ordering.
Go: sqlc regen; MediaMetadata/handler params gain Subtitle/Explicit/
Authors/Publishers (legacy singles still accepted, arrays win);
SeriesNumber int32 -> float64 end to end; metadata_overrides adapted
with legacy override keys ('author'/'publisher') preserved so existing
user overrides survive; bulk author edit writes the array.
Pins widened (docs/contract_pins.md): raw sqlc row key set + the six
new columns; changelog entry added.
Verified: unit suite green; integration sweep (bulk/smart collections/
scanner/stats/sync/queue/calibre/pins/audiobooks) green against the
shared dev DB after in-place migration. Pre-existing unrelated red:
TestOwnershipAndAdminTestSuite/TestAdminCreateUser timezone inheritance
fails identically at the pre-change commit (live shared state).
|