f47734dd7d0b0f619b02fefc665bab28c5219eb9
377
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 John O'Keefe
|
f47734dd7d |
fix(podcasts): generic media delete refuses episode rows (invariant 2)
The admin endpoint DELETE /api/media-items/:id predated podcasts and would delete any media row — including podcast episodes — if hit directly. Invariant 2 (absolute form, user decision 2026-10-30) says episode rows are removed ONLY by whole-feed removal: "the feed should match the XML until the entire feed is gone." The plan audit found this endpoint was the one remaining surface that could violate it: the poller, unsubscribe, and the admin file-deletion endpoints are files-only, and the hash-conflict merge can't see episodes (they never hash). The guard sits right after the GetMediaItem read and BEFORE the storage-down check, so a refused episode never touches storage resolution. Same segregation pattern as the generic edit path (D9/D10, UpdateMediaItem). Pinned by TestDeleteMediaItemPodcastGuard: admin DELETE of an episode → 409 with the row surviving; a book delete through the same endpoint → 204 control. |
||
|
John O'Keefe
|
7d5bbd1177 |
feat(admin): multi-select episode file deletion — checkboxes on downloaded rows
The plan's per-EPISODE bulk select for the admin view (§0 item 8 follow-up — trimmed in the first admin-view commit; user called it back). Admins now tick checkboxes on DOWNLOADED episode rows and delete their files in one request: - Checkbox per row, rendered ONLY for admins + downloaded episodes (nothing to delete otherwise), via the existing podcastDetail Alpine component: selection ids/count survive the list's htmx swaps (reapplied on afterSwap — same pattern as the text filter) and clear after a successful bulk delete. - Toolbar segment outside #episode-list (survives swaps): 'N selected' + 'Delete selected files' (hx-include gathers the CHECKED boxes); disabled-looking at zero selection; hx-confirm states the re-materialization contract. CSP-safe method bindings only (bulkSelectionLabel/Class) — no ternaries in Alpine expressions. - POST /feeds/:id/episodes/delete-files (admin): repeated episode-file-select keys parsed via ParseForm directly (never bound — repeated-key wire is another echo-bind gotcha); each id validated to belong to THIS feed, so a forged foreign id is refused while the real selections apply; rows reset to pending, files removed. - Tests: bulk of two real ids + one forged id → deleted:2, files gone, rows pending; non-admin 403; all prior single/feed-level pins hold. |
||
|
John O'Keefe
|
0741e98ea5 |
feat(admin): all-podcasts view — disk-first maintenance for every feed
§0 item 8 (Phase-4 item 1 pulled forward, user directive: maintenance
tooling ships WITH the features it maintains):
- /admin/podcasts (AdminMiddleware): every feed from EVERY user via the
RLS admin escape (first-admin identity, poller/fetcher contract) —
feed + host, library, subscriber count/names, downloaded/total, DISK
USAGE, last fetch. Sorted disk DESC ('what's eating the disk'),
totals header (feeds · episodes · bytes on disk).
- Per-feed actions: Refresh · Delete files (rows → pending, disk freed;
confirms spell out the re-materialization contract) · Remove feed
(rows + files for everyone — the same admin delete as the detail
page). All confirms ride the global confirm modal.
- New sqlc: ListAllPodcastFeedsAdmin (correlated aggregates),
ResetPodcastEpisodeFileDeleted / ResetPodcastFeedFilesDeleted —
downloaded rows reset to pending with download-derived facts cleared
(audio_metadata, chapter_count, cover, size, mime); file_path KEPT
(fetch-maintained layout — a re-download lands on the same path).
file deletion via the existing DeleteFeedFiles file-removal pattern
(episode files + .cover.jpg siblings).
- Poll upsert never touches download_state, so deleted files STAY
deleted — a feed refresh re-lists metadata but re-materializes
nothing (pinned by TestAdminDeleteEpisodeFiles).
- Per-episode admin delete-file: POST /feeds/:id/episodes/:eid/
delete-file (admin) — one episode's file gone, row pending, siblings
untouched; HX swaps the episode list. (Found while testing: the
handler first read c.Param("id") — the FEED's uuid — instead of
:eid; the 404 'episode not found' trail led straight to it.)
- Tests: admin sees other users' feeds + subscriber names; regular
users 403 on page AND actions; single + feed-level deletion reset
rows/clear facts/delete files; refresh leaves pending rows pending.
- Admin nav gains 'Podcasts' under Administration.
Wrong-gotcha note for future agents: parseUUID on c.Param('id') inside
a /feeds/:id/episodes/:eid route resolves the FEED, not the episode.
|
||
|
John O'Keefe
|
5c545534c2 |
feat(podcasts): Download All queues in the LISTENING order + add-modal order dropdown
Download All previously queued in feed-native order — 'getting the last chapter first doesn't help me'. Now the queue follows the feed's effective listening order (§0 item 5b): QueueFeedDownloadsAs reads the CALLER's order override and runs episodes through services.OrderEpisodes (same helper as the sequence/reader/page surfaces — the consolidation from item 5a landed with the reverse-order commit). Queue tickets (SEQUENCE-stamped, item 4) make that enqueue order the claim order, so chapters land in the order you'll actually listen. Add modal (§0 item 5c): with 'Download all episodes now' checked, a 'Download order' select appears (pure-CSS :checked sibling reveal — no JS island, CSP-clean). Its choice is the feed's INITIAL order override (reader_settings upsert; 'feed' = no row) AND the queue order for the initial batch. Per-book by design: the override IS the download order. Test TestPodcastAddModalDownloadOrder: add with download_all+reverse → the override row exists AND persisted tickets run position 3→1. Tickets are asserted (not live queued rows) because the shared dev server's worker legitimately races this test's queue — tickets survive every state flip. |
||
|
John O'Keefe
|
0f3c4a8cec |
feat(podcasts): concurrent download pool — fair share, tickets, admin tunables
Replaces the sequential worker with a download-manager-style pool (§0 item 4). Model: the user's connection is the limit; semantics are fair-share ACROSS feeds, not goroutine-per-book (a global FIFO would make Book B wait behind Book A's 100 files). - Atomic claim: GetNextQueuedEpisode + SetPodcastEpisodeDownloading was a read-then-write race under concurrency; ClaimQueuedPodcastEpisode converts queued→downloading in ONE guarded statement (two claimants, one winner — the loser gets ErrNoRows). SetPodcastEpisodeDownloading retired. - Queue tickets: download_queue_pos BIGINT fed by a Postgres SEQUENCE stamped at enqueue — the enqueue loop's ordering IS the claim order (the foundation §0 item 5 builds on; previously all rows in a batch shared one updated_at, so within-feed order was arbitrary). - Pool: up to podcast_download_concurrency (default 4, live range 1–16) claim-loop goroutines per drain round; the setting re-reads every round so admin changes take effect without restart. - Fair share: claim prefers the feed least-recently-served (pickFeedToServe; never-served first), earliest ticket within a feed — books proceed in parallel while chapters land in order (start listening while the rest download). - Bandwidth: shared token bucket across all workers (podcast_download_speed_mbps, default 0 = unlimited); take() reserves bandwidth and callers sleep outside the lock. - Settings: new float setting type end to end (registry defaults + clamping, save validation, admin UI number-with-step input) + the two schema seeds; worker wires the registry via SetSettings (tests run on registered defaults). - Tests: unit coverage for the bucket reservation math + fair-share pick; integration TestPodcastDownloadPool pins ticket order = enqueue order, claim exclusivity, and a full 3-episode pool drain. Dev-DB note: the column/sequence/settings were applied live (idempotent DDL, same statements the schema runs at next boot) since the running container predates them. |
||
|
John O'Keefe
|
e3675536aa |
feat(podcasts): reverse order — position DESC, date-agnostic; single order helper
The 'Oldest first' order sorts by PUB DATE, which breaks on Scribl's migrated podiobooks feeds (messy/all-equal dates). New order value 'reverse' = feed position DESC with a date-DESC tiebreak — dates are not involved, so nothing can break. - services.OrderEpisodes: THE single order source of truth (§0 items 3+5a). The four surfaces (sequence JSON, reader chain context, SSR page list) previously carried three near-identical handwritten comparators — all now route through one generic helper. The SSR page list gains a SortDate key (lexical 2006-01-02) so the pub-date tiebreak is actually chronological; the old comparator compared 'Jan 2, 2006' display strings alphabetically (latent drift bug). - Dropdown becomes four options: Feed order / Feed order (reversed) / Oldest first (pub date) / Newest first (pub date). - Serial suggestion button renamed 'Reverse feed order' (sets reverse, not oldest) and its response out-of-band-swaps the order select (PodcastOrderSelectOOB — same mechanism as Download All's policy select), so the dropdown mirrors the server state immediately. - Validation: oneof=native reverse oldest newest everywhere. - Tests: reverse binds via the form wire; messy-dates sequence returns 3,2,1; existing reorder-safety stays green. |
||
|
John O'Keefe
|
7c7c08748e |
fix(podcasts): download_all binds on every wire — FlexBool + flip log
Reproduced the §0 item 2 suspect: echo v5 bool-binds form values via strconv.ParseBool, which accepts "true"/"1" but REJECTS the browser-native <input type=checkbox> value "on" — and a bind failure kills the WHOLE AddFeed request (400 "invalid request"), not just the checkbox. The add modal posts urlencoded (htmx), so any checkbox-serialization that emits "on" made Download All silently unusable. Fix: FlexBool (encoding.TextUnmarshaler-style UnmarshalParam) binds download_all permissively on every wire — "true"/"on"/"yes"/"checked" => true, explicit false/0/off => false, ABSENCE (unchecked box on any wire) => false; JSON bool binding is native and unaffected. Tolerates the exact wires htmx and browsers emit. AddFeed's subscription flip now logs "[PODCAST] flip ... <feed> — user" so the live flip is provable in app logs (user suspected it never fired). Integration test TestPodcastDownloadAllFormEncoding pins the urlencoded wire (what htmx actually sends): true + on both flip the caller's policy to 'all', an unchecked add leaves the default. |
||
|
John O'Keefe
|
7ea84043fa |
fix(auth): logout actually signs out — clear the HttpOnly session cookie
Release / build-and-push (push) Successful in 3m9s
Logout wiped localStorage but never cleared the token cookie (HttpOnly — only a server response can) and never called the logout API, leaving the refresh token live for 30 days and every cookie-authenticated SSR page accessible: the fake-signout bug. Reproduced on prod and dev; unrelated to RLS (it sits behind authentication) and long-standing. - /api/auth/logout now ALWAYS clears the session cookie (Set-Cookie empty + Max-Age=-1, HttpOnly/Path/SameSite mirroring login) — even a malformed body must sign the browser out; refresh-token revocation unchanged. - The header logout button (and the rules-page copy) now POST to the API with the stored refresh token BEFORE clearing local state and redirecting — best effort: local cleanup never blocks on API failure. - toast 401 handler fixed: it cleared 'refreshToken' but storage uses 'refresh_token' — the real key. - Regression test TestLogoutClearsSession: real login → logout → response deletes the cookie; logged-out refresh token can't mint access tokens; body-less logout still clears. Also carries regenerated artifacts from the confirm-modal and episode-list work (header_templ/podcast_detail_templ/style.css). |
||
|
John O'Keefe
|
30766020c8 |
feat(podcasts): real-time episode list — byte progress bars, direct swaps (issues #1/#3)
WHY: downloads needed manual refreshes to see progress; the 204 + HX-Trigger event indirection missed its listener, and the worker buffered whole enclosures so no progress existed to show. - Worker streams enclosures to disk through a counting loop (64KiB chunks, scheme/redirect/size guards preserved) with an in-memory per-episode progress snapshot (received/total; total from Content-Length, seeded by the feed's declared length). Deliberately transient — the DB row state remains the durable truth. - Episode-list renderer seam: the router injects a closure into PodcastHandler (handlers can't import templates — MediaDetail cycle) so htmx download/retry/Download All/refresh/order/policy actions answer with the REFRESHED partial directly (200, hx-target= #episode-list) — no events to miss. Non-HX callers keep JSON; tests without a renderer keep 204+trigger. - Polling tightens to 2s while downloads are active; downloading rows render a percentage bar + MB counter (quote-free inline style per templ gotcha #8) or an indeterminate bar when the server sends no length. - Issue #3: Download All (button AND the add-feed checkbox) flips the CALLER's subscription policy to 'all' — per-user, never global (D6: files shared, policies not) — and the auto-download select out-of-band-swaps so the dropdown shows what the user is getting. They can turn it down afterwards; the flip is honest communication, not coercion (user-approved 2026-10-29). PINs: Download All with HX answers 200 + episode-list + oob select and flips newest → all; add-feed download_all sets policy all. |
||
|
John O'Keefe
|
c753780817 |
fix(podcasts): order selector validation — bind the htmx form wire format (issue #2)
WHY: htmx sends non-GET requests urlencoded by default; the SetFeedOrder request struct only carried a json tag, so the binder left Order empty and 'required' fired — the exact error the order dropdown AND the Serial-suggestion button produced (hx-vals JSON also arrives form-encoded). One form tag fixes both controls. Regression test drives the exact wire format: urlencoded PUT with order=oldest/native → 200, empty/invalid → 400. |
||
|
John O'Keefe
|
2235b00c59 |
feat(podcasts): web UI — /podcasts page, feed detail, download lifecycle (task 1.9)
Rendering ownership holds throughout: every piece of server data is
templ-rendered; htmx swaps partials; the only TS is the Add-Feed modal
toggle + error surfacing (pure client-local state, bundled via
web/src/podcasts.ts with automatic bundle-hash stamping).
Sidebar: /podcasts nav entry (D13, between Series and Collections),
gated by a ShowPodcastsNav flag resolved centrally in
getTemplateUserWithTheme (one indexed COUNT — same class of
per-request lookup as the live-user guard). New 'podcast' icon.
/podcasts: header + Add Feed modal (htmx post → HX-Redirect to the
feed page; inline error surfacing on failure), subscription-scoped
search + facet bar (host/tag/language — server-side GET filters, no
JS), grid cards with download counts / new-episodes / complete /
explicit badges. Empty states for no-library (admin creates it first)
and no-subscriptions.
Feed detail: hero (artwork, host/publisher, tags, fetch state), action
row — Download All, Refresh, ORDER SELECTOR (D5 override), serial
one-click oldest-first suggestion (never silent), auto-download policy
select (D6), Unsubscribe (files retained), admin Remove Feed — gap
banner (D8 count), and the EPISODE LIST with the full lifecycle inline
(D8/D19): pending greyed + Download button (every non-downloaded
episode has one), queued/downloading pulse, failed RED + reason text +
Retry (same endpoint). The list is a self-refreshing partial: re-fetch
on the podcast:changed HX-Trigger and polls every 4s ONLY while the
server says downloads are active (it renders the polling variant).
API endpoints grew HX awareness: 204 + HX-Trigger podcast:changed (or
HX-Redirect for add/unsubscribe/remove) — the sanctioned Go-handler
response channels; JSON callers keep JSON. AddFeed result exposes the
fetch-error text for the redirect.
Dates render in the requesting user's timezone (D17) via
utils.FormatTimeInTimezone; episodes play through the existing
/readers/{id} audio reader (episodes ARE audiobook items — no new
player code day one). CSP tripwire (templates/csp_source_test.go)
passes: no inline scripts, no on*= handlers, no hx-on/js: vals.
|
||
|
John O'Keefe
|
54d7d6e7a1 |
feat(podcasts): edit segregation — feed metadata is fetcher-owned (task 1.8)
D9/D10: users never edit podcast episode metadata. Three gates: - Generic edit endpoint (UpdateMediaItem) rejects source= 'podcast_episode' items with 400 — the next fetch refresh would clobber any edit anyway, and the fetcher is the sole metadata writer. - Write-to-file (MetadataWriter.WriteMetadataToFile) refuses podcast episodes for the same reason (tag write-back for downloaded episodes is a named Phase-4 item needing its own iTunes-tag mapping). - The book-detail page hides the admin Edit affordance for podcasts-type items (direct-URL visits), and the metadata-editor modal's field list has no podcast fields by construction (fixed templ field list) — episodes render read-only via /podcasts surfaces. |
||
|
John O'Keefe
|
45b1152a15 |
feat(podcasts): API endpoints — add/list/detail, subscriptions, downloads, sequence, search (task 1.7)
The §8 surface, wired through main.go → router.Config → /api/podcasts (JWT). Scoping is the engine's (D20): lookups outside the caller's scope simply return no rows → 404; the only Go-level checks are explicit-argument authorization (AddFeed's library + allow_member_feeds gate) and the admin role on feed deletion. Endpoints: POST/GET /feeds (add: fetch+parse+upsert+subscribe, optional download_all; list: subscriptions + episode counts + facet values computed in Go — feed counts are tens), GET /feeds/:id (detail with download states + gap count), PUT/DELETE /subscription (the ONLY user-writable podcast state, D9; unsubscribe retains files per D6), POST /refresh, POST /episodes/:eid/download (+ episode_ids[] bulk and eid='all' = Download All, exactly the non-downloaded set), PUT /order (D5 per-user override in the reader_settings KV — podcast_order:<feed_id>; 'native' deletes the row so default and explicit choice are indistinguishable), GET /sequence (effective order — override applied, native fallback — DOWNLOADED-only with the gap list; this payload powers web chain mode and, per §15, the future media3 queue), GET /search (subscription-scoped feed + episode search covering un-downloaded and historical rows, D14), DELETE /feeds/:id (admin; best-effort file removal, cascade deletes rows). Gap definition (countGaps): an un-downloaded episode with a downloaded episode AFTER it in list order — the D8 'middle episodes missing' banner condition. |
||
|
John O'Keefe
|
9a148c2344 |
feat(podcasts): generic-surface exclusions — podcasts never render as books (task 1.3)
§9 / D3 / D13 / D15 / D16: podcasts-type libraries are excluded from every generic book surface. Under RLS these are pure UX/type filters (engine already hides other users' episodes); their job is preventing episode grids in surfaces designed for books. The discriminator is the denormalized media_items.library_type_name (IS DISTINCT FROM 'podcasts' — NULL rows predate the column and stay visible). SQL (queries.sql, marked '-- podcasts exclusion (plan §9)'): ListMediaItems/ListMediaItemsSorted (All Books) · SearchMediaItems + SearchMediaItemsUnified (generic search) · FacetCountsUnderFilters (all 14 arms) + the six Search*Values facet-suggestion queries · GetDistinctSeries + Count (series pages) · GetLibraryItems (collections picker) · dashboard system sections (Continue Reading / Recently Added / Recently Read / Not Started) · GetPopularBooks + GetTopBooksByMinutes (book rankings must not rank episodes). Go: statistics summary counts episodes' sessions into time-spent (D15 keeps that) but skips them in BooksActive/BooksFinished — an episode is not a book. OPDS filters podcasts libraries via one helper at all 7 library-walk sites (D16). Templ: LibrarySwitcher skips podcasts entries and TotalMediaCount excludes them — presentation-layer filter so every switcher consumer (dashboard, collections, custom-section builder) is correct at once (D13); podcasts surface via /podcasts only. |
||
|
John O'Keefe
|
c0e6993273 |
feat(podcasts): scanner exclusion — podcasts libraries are feed-managed (task 1.2)
WHY: podcast download roots hold fetcher-written episode files; the FS
walker must never discover them (invariant 4) — a walk would double-
ingest episodes as scanner items and fight the fetcher's sole-writer
rule (D9). Every scan/watch path funnels through MediaScanner.SetFolders
(StartScanner, ScanLibrary jobs incl. scoped single-folder scans,
watch mode), so the filter lives there once: folders resolving to a
podcasts-type library are dropped before registration, with a log line.
Belt and braces: ScanLibrary's per-folder scope refuses purpose='podcast'
folders explicitly (400) — the panel-level path that bypasses type
resolution. AddLibraryFolder now derives purpose from the library type
('podcast' for podcasts libraries, enforced single download root;
'scan' default elsewhere), so folder purpose and library type cannot
diverge through the API.
|
||
|
John O'Keefe
|
d49a9e6ca2 |
fix(devices): device deletion detaches reading history instead of 500ing
Deleting any device whose stats rows still referenced it failed with 500: reading_history.device_id is a NO-ACTION FK by design (the rows are real captured reading stats — Users reads them), so a blind DELETE violated the constraint. Surfaced by the Android app's new logout deregister call, but the admin Devices page Deletes hit the same stone. Fix: DetachReadingHistoryFromDevice — NULL the device association on the user's history rows before the device DELETE (single handler pass). device_id is nullable; the stats survive, the device frees its identifier. sqlc regenerated. |
||
|
John O'Keefe
|
4d697f5fcf |
test(visibility): full suite green under RLS — sync paths, device-auth bridge, seed pool
Phase 5 (plan §7): the entire integration suite passes with the RLS policies live. The suite was the triage enumerator the plan predicted: - device_auth middleware: the THIRD identity bridge — KOReader/Kobo device-authenticated requests read media as the DEVICE OWNER (plan A9 called this path out; the flip was 404s on sync metadata). - conflicts/analytics/progress handlers: detached context.Background() reads threaded to the request context (the requesting user's identity) — 12+ sites; resolve helpers take ctx. - scanner: ScanFolders self-declares its configured admin identity (server maintenance, product-correct for any caller); generic job creation stamps the requesting admin. - harness: setup.DBPool and the sync-test pool declare rls.SeedIdentity at SESSION scope via BeforeAcquire — the pool never serves a second identity, so no A3 bleed; setup.DB falls back to the seed identity (NewSeedDB) for ungoverned-context calls; filter tests use seedExec for raw governed-table statements. - contract pin WIDENED (docs/contract_pins.md 2026-10-29): under RLS, item-scoped gates may answer 404 (row invisible — no existence leak) as well as 403 (Go-level checks on listings keep strict 403). - TestWorker_JobStatusTracking — the pre-existing red from the Phase 0 baseline — passes with the harness seed work; no product change. |
||
|
John O'Keefe
|
bb2cc26e9d |
feat(scanner): scanner entrypoints + generic jobs carry RLS identity; SeedDB harness fallback
The full suite enumerated the remaining triage class: tests drive the scanner directly (ScanFolders(context.Background())) and read seeded media via Queries with bare contexts — every governed read raised. - MediaScanner.ScanFolders self-declares its configured admin (SetAdminID — set by main AND the test helpers): a scan is server maintenance, product-correct regardless of caller. - ScopedDB gains a Fallback identity, exposed as NewSeedDB for the TEST HARNESS ONLY: setup.DB falls back to rls.SeedIdentity when the context carries none, so direct seeding/verification helpers work uniformly instead of per-site context threading. rls.SeedIdentity is the shared constant (no library_visibility rows => full visibility). - generic job creation (jobs.go) stamps the requesting admin into the job Context, same as the scan endpoint. The sync queue processor stays identity-free: sync_queue is not an RLS-governed table. |
||
|
John O'Keefe
|
385a5d9dba |
feat(scanner): background jobs declare acting identity for RLS
Phase 4 of the RLS plan: detached contexts (context.Background()) that reach media_items now carry identities — with the policies live, a bare context made governed reads raise (fresh connections: 42704 unrecognized parameter; reused pool connections: 22P02 empty-string cast after the LOCAL GUC reverted — the plan's A2 signature). - scanner jobs (API-triggered, mtime-poll, initial scan): the job's Context now carries the acting admin's identity — the library owner, first-admin fallback (the job's UserID already encoded this trust decision; the Context just needed to carry it). performInitialScan uses the scanner's configured admin. - websocket initial-sync read (GetUserMediaItemsForSync): runs as the connection's own user. - hash backfill: runs as first admin (server self-heal over all media). Verified on the dev stack: forced rescan of the 1199-file Audiobooks library completed with 0 errors under RLS (previously every SHA lookup and create raised); hash-backfill refusals gone. The sync queue processor needs no identity — sync_queue is not an RLS-governed table. Known remaining detached reads (conflicts/analytics handlers using context.Background()) are HTTP-path triage, enumerated by the full suite next. |
||
|
John O'Keefe
|
9f39e2efd8 |
feat(settings): save feedback — success/error toasts, disabled buttons; fix scan-form raw-JSON swap
Release / build-and-push (push) Successful in 4m35s
None of the three save surfaces on the admin settings page gave usable
feedback: the Base URL form re-rendered into an identical-looking form
(its only indicator was the Go-source twin's banner — markup that
diverged from the templ original), the scan form was WORSE — its handler
responds JSON, so htmx's outerHTML swap replaced the whole scan card
with literal JSON text — and the tunable rows showed only a tiny
per-row label.
Client side (toasts are the sanctioned client-local island; no new
inline scripts, CSP unchanged):
- web/src/settings.ts: one delegated htmx:afterRequest/sendError
listener keyed on a data-settings-save attribute stamped on each
form. Success → green toast per surface ("Settings saved" / "Scan
settings saved" / "Setting saved"); failure → error toast carrying
the server's JSON error field, degrading to a status line for the
tunable endpoint's HTML error body (markup must never reach a toast).
Network-level failures (htmx:sendError) toast instead of silently
re-enabling the button.
- htmx 2.x quirk handled: for FORM submissions, afterRequest's
detail.elt is the event-dispatch ancestor, not the form — the
requesting element lives on detail.requestConfig.elt (verified
empirically; saveLabelFor reads the right one).
- hx-disabled-elt="this" on all save buttons: the click is
acknowledged and double-clicks cannot double-save.
Server side:
- Scan form: hx-swap="none" (the inputs already hold the saved values;
nothing needs re-rendering) — kills the JSON-over-the-card swap.
- The Base URL form's save response is a Go-source copy of the templ
form (handlers cannot import templates: templates/utils.go imports
handlers types). The twin now carries the functional attributes
(data-settings-save, hx-disabled-elt) so feedback survives the swap,
with a drift-warning comment; the double-save e2e pins this.
Dev-DB safety: every success save in the spec re-submits the value
already on the page (zero drift), and the spec restores
system_config.updated_by after itself — the save handler stamps the
requesting admin's id there, and that FK (no cascade) otherwise blocks
the integration harness's transient-user sweep (it broke
TestUpdateProfileAdminMode/TestDeleteUserConsolidated on this run's
first pass; pollution cleaned up by hand and now self-restoring).
Tests: web/src/settings.test.ts pins the form→label map and the
error-extraction contract (including the no-markup-in-toast rule);
e2e/settings-feedback.spec.ts drives all three surfaces through the
real admin UI including the error path and the twin-drift double-save.
|
||
|
John O'Keefe
|
96c1228805 |
feat(series): multi-field search, sort-direction toggle, apply placement
Toolbar feedback round: the series search box now matches whatever the main search matches for a group — series NAME, any book's AUTHOR, and any book's TITLE inside the series (WHERE EXISTS per group-row, so counts stay true); the separate author input is gone from the toolbar (the author= param stays live for the JSON API and deep links). The sort control becomes a base-key select (Date added/Name/Author/Books/ Progress/Last read) plus a chevron direction toggle that flips and resubmits; sort/dir compose through ResolveSeriesSort (full registry keys keep working for back-compat, unknown base keys fall to recent). The search/apply button moved next to the search box as requested, and default URLs stay clean (sort/dir only emitted once the user chooses). |
||
|
John O'Keefe
|
eaf1b2c8dd |
feat(metadata): write-to-file — embedded EPUB OPF + Calibre sidecars
Phase 5a of the metadata-writer feature: the book editor gains a "Write to file" action (admin) that persists the item's CURRENT database metadata into the artifact on disk, so the library stays portable to Calibre and other apps — the thing Save alone can't do. Dispatch (services/metadata_writer.go): EPUB/KEPUB gets its embedded OPF spliced in place (fresh dc:* block generated inside the ORIGINAL metadata open tag; manifest/spine/guide untouched; every other zip entry copied byte-identical in original order so mimetype stays first and STORED; original dc:identifiers preserved — the package's unique-identifier must keep resolving). Everything else takes a Calibre-style sidecar: metadata.opf when the book is ALONE in its directory (Calibre's own convention — round-trips through Calibre too), <basename>.opf when siblings share the folder, so one book's sidecar can never poison dir-mates' scans (the scanner now reads both, basename first). Safety contract: temp file + atomic rename — the original is never truncated on failure; the generated OPF is unit-tested to ROUND-TRIP through the scanner's own parser (write side and read side can never disagree); metadata_overrides are deliberately kept (file and DB agree after the write; overrides still guard against stale sidecars). The endpoint re-extracts inline (RescanMediaItem) and reports rescan failure separately — a failed rescan doesn't undo the write. SHA-256 changes are inherent and documented for users (Kobo/KOReader re-key). UI: the metadata editor modal carries the file path as a data attribute; the button confirms before mutating, labels the target (embedded vs sidecar) and toasts the outcome. Integration tests pin the 403, missing-file error, EPUB embed + post-write rescan title, the lone-book metadata.opf case, and the shared-dir <basename>.opf case with no dir-scoped litter. |
||
|
John O'Keefe
|
61e5f47ec0 |
feat(search): viewer progress + completed badges; dropdown to templ
Search results now show where you are in a book: a completed check at the 0.999 threshold (same cut as the filter status arms), a partial progress bar along the cover while mid-read, nothing when you never opened it. Rendered in BookCard (any grid whose adapter loads viewer progress) and in the header-search dropdown rows. The progress rides a LEFT JOIN reading_progress on the SAME user_id param SearchMediaItemsUnified already binds for library visibility — one requester param, no new wire inputs, row-neutral join (the table is UNIQUE per media_item+user). SearchMediaItemsUnifiedRow and BookInfo gain the keys additively (viewer_percentage null when unknown, library_name empty on feeds that don't load it); contract_pins.md carries the changelog entry and the pin tests widen in lockstep. The header dropdown itself migrates off the innerHTML string renderer in search.ts (a rendering-ownership violation: a templ twin existed via the HX-Request branch). The input now hx-gets layout=dropdown and swaps the new templ SearchResults partial — titles/authors highlighted server-side (escape-first, safe by construction), rows with the same progress overlays — while a slim Alpine component owns only client-local state (min-length gating, open/close, keyboard selection, htmx lifecycle). web/src/rendering-allowlist.txt drops the search.ts baseline to zero accordingly. Integration test pins the per-user join (fresh user sees 1.0/0.42/null on the fixture; the shared admin sees null everywhere — read-only) and the dropdown partial's overlays/count/row count. |
||
|
John O'Keefe
|
d1f9ad3b25 |
feat(series): sort and filter the series page
The /series page rendered its grid in one fixed order (newest first)
with no way to find a series beyond eyeballing cards. The page now has
a toolbar — series-name search, author filter, viewer-progress filter
and a 12-key sort select — wired as a plain GET form (the page's
existing model: the URL is the single source of truth, and the library
switcher now carries q/author/status/sort along on switch).
Extensibility is deliberate (user request): sorts live in a registry
(services.ResolveSeriesSort) mirrored by a CASE cascade in
GetDistinctSeries and a <select> in series.templ — a new sort is one
entry in each. The sort value is a BOUND PARAM compared inside the
cascade, never interpolated; unknown keys resolve to the default.
Filters: series-name matches in WHERE (the group key, so counts stay
true); author (BOOL_OR any-book match) and status (finished/reading/
unstarted vs the viewer's progress, same 0.999 threshold as the
filter_query status arms) in HAVING so COUNT(*) keeps reporting the
whole series. GetDistinctSeriesCount shares the JOIN + HAVING so
pagination totals match. The reading_progress join can't multiply rows
(the table is UNIQUE per media_item+user).
Cards gain the viewer's finished-count ("X of N read" + progress bar),
and the empty state distinguishes "no series at all" from "no matches —
clear filters". The JSON /api/series endpoint takes the same params and
gains read_count/last_read_at additively.
Integration test pins every sort key against a three-series fixture
with distinct created/author/count/progress answers, the filter matrix,
composition, unknown-sort fallback, card read-counts and the JSON
contract — all as the fresh regular user so the live reader's data is
untouched.
|
||
|
John O'Keefe
|
c71ef4af85 |
fix(auth): show error feedback on failed htmx login/registration
Found while double-checking /login after the redirect fix: a wrong password left the form completely silent. Two stacked causes, both pre-dating the CSP work: 1. The two HX-Request branches for invalid credentials returned JSON, unlike every sibling error branch — there was no HTML partial to show at all. They now return a shared invalidCredentialsHTML div (the JSON shape is kept for non-htmx API callers). 2. htmx does not swap 4xx/5xx responses by default, so even the HTML error partials that DID exist (429 lockout, 500s, registration failures like "Email already exists") never landed in the target. auth_forms.ts now swaps them itself on htmx:beforeSwap when the error response is text/html — JSON errors stay ignored. Verified live: /login success → /dashboard; wrong password shows "Invalid email/username or password." and stays on /login; duplicate email on /register shows "Email already exists". Gates: make test 9 pkgs, vitest 155, Playwright 20. |
||
|
John O'Keefe
|
7c9e561e68 |
fix(auth): login/registration forms redirect again under hash CSP
The htmx login/registration partials answered success with an inline <script> that stashed the auth token and set window.location — refused by hash-mode script-src (no 'unsafe-inline'), so the user saw "Login successful! Redirecting..." and nothing else (reported from the home- page login form; /login was equally broken). The same pattern lived in collections.go (delete → redirect) and devices.go (token regen → reload, whose script tag was also unclosed). Replacement, per the AGENTS.md CSP rules: - auth.go login/registration: the partial now carries data-auth-token + data-redirect (redirect is path-only — open-redirect hygiene — and escaped); web/src/auth_forms.ts (new, armed from main.ts on every page) listens for htmx:afterSwap, stashes the token, and navigates. - collections.go delete: data-redirect, same listener. - devices.go token regen: htmx's native HX-Refresh response header. - The dead localStorage 'user'/'refreshToken' writes are dropped — no reader anywhere in web/src; only 'token' is load-bearing (19 modules). Tripwire extension: internal/middleware/csp_go_sources_test.go scans non-test Go sources for literal <script> tags and native on*= attributes (hand-written files only — templ codegen is owned by the .templ scan), verified to fire on a reintroduced probe. AGENTS.md CSP section notes the Go-emitted-partial vector. Verified live: home-page login form → /dashboard with token stashed and dashboard loading clean; /register form → /dashboard; collections and devices share the same mechanisms (data-redirect / HX-Refresh). Gates: make test (9 pkgs incl. both tripwires), vitest 155, Playwright 20, tsc/vp clean. |
||
|
John O'Keefe
|
0f1d0abbb0 |
feat(libraries): scope a library scan to one registered folder
POST /api/libraries/:id/scan accepts an optional {"folder_path": "..."}:
the scan then covers exactly that one registered folder's subtree instead
of the whole library. The path is validated against the library's
registered folders — anything else is a 400, never trusted. An omitted
body keeps today's full-library behavior, and coalescing (409 with the
active job) is unchanged.
The LibraryPanel folder rows get a per-folder Scan button: pure htmx
attributes posting to the same scan endpoint, with hxValsFolderScan
emitting plain eval-free JSON for hx-vals alongside the existing
hxValsFolderDelete precedent. Progress surfaces through the existing
active-scan polling; no new client code.
Safety: the scanner's archive-reconciliation guard (previous commit)
skips partially-covered libraries, so a one-folder scan cannot mark the
library's other folders' items missing.
|
||
|
John O'Keefe
|
6f37be59fc |
feat(issues): retry a processing issue through the individual scanner
POST /api/libraries/:id/issues/:issueId/retry (admin, family-lens gated via the existing gateIssue) re-runs the work that produced an open issue — deliberately NOT a library-wide rescan. Two shapes mirror the two ledger entry kinds: - item-backed issues rescan their media item through the book-detail RescanMediaItem mechanism (self-resolving folders, multi-file audiobook routing, write-after-success guards) and then close the issue (RetryMediaItemIssue); - row-less import_failed issues re-import the single file through the watcher's standalone processMediaFile path (RetryFailedImport: folders resolved RescanMediaItem-style so paths stay library-root-relative), whose create-path self-heal plus an explicit path resolve closes the issue even when the content was already imported by a prior attempt. Failures return the REAL underlying error so the UI can show why the issue still exists; a vanished file is a 409 (ErrRetryFileMissing); already-resolved issues reject re-retry. Docs + bruno mirrors regenerated for the new route (the generator's seq renumbering touches three unrelated bruno files as fallout; openapi gains the path and the date bump), and the docs schema registry gains the retry entry so TestDocsMatchRoutes passes. |
||
|
John O'Keefe
|
c645b8e207 |
feat(profile): keep_screen_awake user preference (default on)
New users column surfaced as an On/Off dropdown in the profile form. The form and API carry "true"/"false" strings with the theme-style ""-guard, so JSON clients that omit the field can't clobber it; anything else is rejected. GET/PUT /api/auth/profile expose keep_screen_awake alongside the other profile fields, and the reader route threads the preference into its template metadata. Integration tests pin the schema default, both values, omission semantics, and rejection — dedicated test user, never testuser. |
||
|
John O'Keefe
|
bdb9132b85 |
chore(auth): drop dead UpdateProfileRequest; rename shared struct
The combined profile handler (self-edit PUT /api/auth/profile and admin PUT /api/auth/profile/:id) binds one struct for both paths — and it's the admin-named one. Its twin UpdateProfileRequest had zero references left. Delete it and rename AdminUpdateUserRequest to ProfileUpdateRequest so the name matches who actually binds it; the comments describing field semantics now stand on their own instead of pointing at the deleted twin. Wire format (json/form tags) unchanged; docs schema registration follows the rename. |
||
|
John O'Keefe
|
d41c3e4774 |
fix(ui): hide Comic/Manga editor section for non-comics media
The metadata editor's Comic/Manga section (Manga Type, Reading Direction, Age Rating, Story Arc, Imprint, Scan Info, Metadata Notes, B&W) rendered for every item — an audiobook showed a manga dropdown, which is nonsense. MediaDetail now carries LibraryTypeName (resolved via GetLibraryType in the /media/:uuid handler) and a ShowComicMetadata() gate: comics and manga libraries always show the section; other media only when the scanner recorded comic-style values (fixed-layout manga-style EPUBs get those scanner defaults deliberately — when present they stay editable). |
||
|
John O'Keefe
|
a716b386a1 |
fix: wire security headers, graceful shutdown, and settings invalidation
Lifecycle (internal/app + main.go): - App.OnShutdown hooks run after the HTTP server drains, in registration order: Worker.Shutdown (background jobs now drain instead of dying on SIGTERM), DeviceAuthMiddleware.Stop (releases the rate-limiter ticker), Handler.CloseScanner (fsnotify watcher and scan log files) Security headers: - SecurityHeadersMiddleware (survivor of middleware/security.go, renamed security_headers.go) is now on the echo stack: nosniff, frame-deny, HSTS, CSP (unsafe-inline required by the sanctioned theme flash guard + Alpine), referrer and permissions policy Scanner settings cache: - settings saves (UpdateScanSettings + ApplySetting for the three scan keys) now invalidate MediaScanner's 30s settings cache via SystemSettingsHandler.SetScanSettingsChanged, so changes apply on the next poll instead of after the TTL - MediaScanner.Close closes the scan logger, fixing a file-descriptor leak on every per-item rescan (fresh scanners never closed logs) Client fix: - custom-section-builder's add/remove-selection buttons used inline onclick attributes referencing module-scoped functions, which are not global in a bundled ES module: the buttons threw ReferenceError. Functions restored and exposed on window via declare global Also fixes a pre-existing vet error (saveCoverImage copied echo.Context by value, mutex included) and drops the per-key SettingsCache.InvalidateKey variant left callerless by the full-flush invalidation wiring. |
||
|
John O'Keefe
|
8631aa00bf |
refactor: remove dead code across server, templ pages, and client modules
Deadcode-verified removals (RTA reachability from main, cross-checked with -test and textual xrefs): Go server: - whole files: middleware/transaction.go, middleware/error_handler.go, middleware/password_validator.go (live registration machinery kept), handlers/sync.go, sync/offline.go - unrouted endpoints: auth UpdateUsername/UpdateEmail, timezone settings handler, OPDS RegisterOPDS, Kobo LibrarySync handler, ReaderHandler.ShowReader, JSON-data twins on collections/library - duplicate book-matching handlers (book_matching.go Handler twins, matching.go MatchingHandler twins), superseded helpers (router.parseUUID, handlers.ValidateDeviceToken, normalizePath, userExists, generateLockID) Templ/client: - never-routed pages: api_explorer.templ, queue.templ and their client modules api-explorer.ts, api-explorer-docs.ts, queue.ts - fully orphaned wrapper modules events.ts, dom.ts - broken clearSyncQueue (called a nonexistent endpoint) and the hidden, never-populated sync-queue section on the devices page - 166+ unused exports demoted to private (knip-verified); ~85 truly dead declarations removed via tsc noUnusedLocals fixpoint - stale 'interactive API Explorer' claims dropped from docs The docs drift-gate package (internal/docs), templ-bound modules (dashboard, stats, header, index, login, profile*, bookshelf, collections, toast-error), sqlc-generated code, and web/static are intentionally untouched. |
||
|
John O'Keefe
|
8733e83f1f |
style: apply gofmt across Go sources
Repo-wide gofmt sweep: const-block alignment, doc-comment spacing, statement-after-brace fix in media.go, trailing blank lines. No semantic changes — whitespace-insensitive diff is empty for every file in this commit. |
||
|
John O'Keefe
|
c4b62af213 |
docs(devices): Work Order 3 batch 1 — 6 missing device endpoints + gate entries
New frontmatter'd docs with handler-compatible examples: - devices/get_device.md (DeviceInfo shape) - devices/regenerate_device_token.md (token + sync_urls) - devices/get_device_mappings.md (wrapper + DeviceMappingSummary) - devices/update_device_mapping.md (UpdateDeviceMappingRequest) - devices/download_sidecar_config.md (sidecar config JSON file) - devices/link_book.md (LinkBookRequest; one UNVERIFIED marker on required-tag) Schema registry: 6 new entries; DeviceMappingSummary hoisted to package scope. Gate passes (the UNVERIFIED marker is logged, not a failure). Excludes /api/devices-page (htmx page route, per ground rules). |
||
|
John O'Keefe
|
cc893f758e |
feat(docs): drift gate — docs frontmatter validated against real routes
Every API doc that declares frontmatter (method/path/auth/area) is now checked by a unit test against three sources of truth: 1. Route registration: internal/router/*.go is parsed the way echo resolves it (group prefixes concatenate; scanner.go's parameterized groups seeded). Doc paths that no longer exist fail. (This is the check that would have caught the /api/api/admin/hash-conflicts split.) 2. Auth: frontmatter auth label must match the middleware chain (none|jwt|jwt+admin|device-token|optional-jwt). 3. Shapes: internal/docs/schema.go registry — example-request decoded with unknown fields DISALLOWED (typo guard), required-by-validate fields must appear (staleness guard), example-response top-level key set compared both directions against the marshaled handler type (or a RawMessage pin for hand-built map responses). Five collections docs are the pilots with real handler shapes; notable corrections they pin: update returns 201 (not 200), list excludes smart collections unless include_auto=true, rule priority is 1-10 higher-wins (docs claimed 1=highest), created_at is Go time.String() not RFC3339, hide_finished/show_up_next toggles documented. CollectionResponse hoisted out of GetCollections/GetBookCollections as CollectionSummary / BookCollectionSummary so the registry can pin real types. The gate fails on missing registry entries by design — that is the maintenance contract. Known pre-existing (not this commit): vet copy-lock warnings in media.go. |
||
|
John O'Keefe
|
3576a4415a |
feat(audio-player): per-book playback speed + user default (plan §8)
Speed becomes a per-USER-per-BOOK preference that syncs across devices, with a user-level default for books not yet played: - new audio_playback_speed table (mirrors reading_speed's shape; a preference, NOT a measurement and NOT a reading_progress column) + users.default_audio_speed, schema in place - chapters endpoint resolves per-book row → user default → 1.0 and ships playback_speed + default_playback_speed (zero extra round trips; default powers the dropdown marker) - PUT /api/media-items/:id/audio-speed upserts (range-checked; the client SPEEDS list stays the real gate; library visibility gates writes) - the user default rides the profile path like theme/timezone: profile form gains a 'Default Playback Speed' select (server-side SpeedOptions mirrors the web SPEEDS list), GET /profile carries it - web: restorePosition applies the resolved server speed (localStorage audio_speed demotes to a first-paint cache); setSpeed PUTs fire-and-forget with a failure toast; the Speed dropdown gains a '(default)' row marker and a 'Make N× my default' action with a success toast ALSO fixes a pre-existing profile-page bug surfaced by the §8 form verification: the password section's current_password input was required INSIDE the shared form, so HTML5 validation blocked every Save Changes submission (dead since the shell migration). The password flow keeps its own gates (disabled-button checkers + server 400). Integration test TestAudioPlaybackSpeed pins the full contract (unset → 1x, upsert resolution, range rejection, default fallback for row-less books, default updates re-satisfying unset books). Verified live: 9 Playwright checks. |
||
|
John O'Keefe
|
35d8ba32e5 |
fix(media): metadata editor save no longer wipes audio_metadata
PUT /api/media-items/:id (the metadata editor) writes the FULL row via UpdateMediaItem, whose query sets audio_metadata unconditionally — and audio_metadata has no editor field, so the params literal left it nil and every save NULLed the payload. A plain metadata edit (genre tweak, title rename) dropped any audiobook back to "1 second long" with no track manifest: the same incident class as the scanner wipe, sitting on the most-used admin path in the app. Carry existing.AudioMetadata through the write. Write-after-success: only the audio chain (probe-backed extraction / UpdateMediaItemAudioBookPayload) ever replaces that column. |
||
|
John O'Keefe
|
9b5defcb31 |
feat(issues): the honest processing-issues ledger — row-less import failures, self-healing resolution, family lens (faceted plan Phase 11)
processing_issues only recorded cross-library move rejections; files that failed to import vanished into an error counter and stdout. The ledger now records every failure class and heals itself: Schema (in place, pre-release boundary): - media_item_id NULLABLE + file_path TEXT (deliberately unbounded — an over-long path is itself a failure mode; a VARCHAR(500) ledger column died on the exact 22001 it existed to report — caught by the new test). Row-less failures key on (library, file_path); item-backed on (item, type). Widened dedup = COALESCE-expression unique index; the old table UNIQUE stays (NULLs pairwise-distinct for row-less). - Conflict-target upsert: repeat failures UPDATE the same row and REOPEN resolved ones — never duplicates. Wirings: - CreateMediaItem failure (single-file + multifile createAudioBookItem) → row-less import_failed (error). - Extraction hard-fail with fallback import → item-backed metadata_extraction_failed (warning). - Change-path re-extraction failure (the ev/sidecar guard) → item-backed metadata_extraction_failed; LogProcessingIssue keeps its signature, a row-less helper joins it. Auto-resolution: - Successful import → ResolveOpenProcessingIssuesForPath (library, file_path). - Successful (re)extraction → ResolveOpenProcessingIssuesForItem (single-file change path, multifile editorial apply). - Unchanged-skip scans do NOT resolve: extraction never re-ran, so the degraded metadata is still degraded — the page heals exactly when the book does. Family lens on ALL issues surfaces (list, stats, resolve, delete, SSR page): explicit out-of-set library id → 403, never silent. This tightens one pre-lens behavior — a nonexistent library id returned 200-empty, now 403 (pinned, changelog entry in docs/contract_pins.md). List is LEFT-JOINed so row-less rows appear (media_item_id "", title falls back to the path). Tests: full lifecycle state machine through real scans (fail → appears, repeat → no dups, success-at-path → resolved, LEFT-JOIN list shape); family lens 403s with a fresh non-shared admin; all 7 pre-existing issue suites green (two expectations widened to the lens, changelogged); adjacent suites (CrossLibraryMove, Audiobook, Calibre, ExtractorVersion, SchemaIdempotency) green; unit suites green. |
||
|
John O'Keefe
|
9c006a9a2f |
fix(filters): browse with no rules works, popup under the input, no sticky headers (user report)
- Blank typed queries rode the STATIC autocomplete path for the classic six fields, where a single space similarity-matches into nothing — clicking an empty value box with no other filters said 'No matching values' in a library full of values. Blank = browse now always rides the composed engine (which trims and answers the top values); static keeps its pinned contract for real typed queries. Regression-pinned: browse with no rules lists the field's values. - The value popup now drops BELOW the input (top: 100%), not over it. - Field-menu group headers lost position:sticky — the search box alone stays pinned. |
||
|
John O'Keefe
|
4ee24e7152 |
feat(filters): the rule builder — composed engines over the field registry (faceted plan Phase 9)
- Field registry (services/filter_registry.go): ~55 typed fields — every book column, file/lifecycle facts, the three HOLDING STRUCTURES (library, collections with excluded-aware membership, series with FULL primary+alternate_info membership), and the whole user-relative class (status, progress, last read, my rating, review, notes/highlights presence, note/highlight text, time spent, pages read, collections, added by) reading the VIEWER's rows. Operators are typed constants; negation is an operator. - Composed engines (services/filter_query.go): one Go clause builder over raw pgx feeds the search (identical SearchMediaItemsUnifiedRow shape), the field-aware suggestions, and a true-COUNT live count. Mandatory WHERE head everywhere (family lens + integrity facts). Values are always bound parameters — no user string reaches a field/operator position, ever. - Wire: the qf JSON param (validated strictly — unknown fields/operators/malformed values are 400); qf wins over flat params on hand-crafted URLs; flat deep links keep working unchanged. - Endpoints: GET /api/media-items/filter-count (live count); field-values autocomplete upgraded in place — any registry field key, qf binds the OTHER rules (symmetric under-filter suggestions, zero-match values never offered); classic six without a tree keep their static pinned contract. - UI: the drawer becomes the rule builder — the ONLY filter UI (templ markup, Alpine tree state, sanctioned island; grouped field menus, operator menus with red negations, field-aware value menus, live count, chips per rule with red negations, URL rehydration, smart serialization flat-when-simple else qf). - Tests: composed semantics (AND/OR/negation/nesting/enums/bools/ numbers/dates/empty), every user-relative field class with the my-data weld (two readers, different worlds), note-text substring pin (catharsis ≠ Carthage), family lens (hidden libraries nowhere, 403s), suggestion-under-rules exactness, qf validation + precedence, flat-vs-tree equivalence; registry completeness unit test. |
||
|
John O'Keefe
|
ebb7f4a5b0 |
feat(filters): panel removal + All-Books field-values fix (faceted plan Phase 8)
- Remove the Phase 7 facet-panel UI: SSR route, facet_panel.templ, FacetCountsForView, drawer panel block, panel TS + .facet-* CSS. The ENGINE stays (FacetCountsUnderFilters, all-facets endpoint, drift guard) — it becomes the Phase 9 builder's machinery. - Fix the All-Books bug: field-values autocomplete (author/genre/ tags/narrator/series/language) treats library_id as OPTIONAL — absent aggregates every VISIBLE library (family lens in-query); explicit out-of-set library_id is now 403, never a silent pass. The four scalar Search*Values queries get the nullable shape. - toast.ts: suggestion fetches are exempt from error toasts (console-only) — a failed background hint never interrupts. - TestFacetPanelPartial deleted with the behavior; TestFieldValuesAll- Libraries pins the fix (200 + cross-library aggregation, family lens, 403). Contract-pins changelog entry added. |
||
|
John O'Keefe
|
f52bb6f073 |
feat(facets): all-facets panel with under-filter counts (faceted plan Phase 7)
The 9 Batch A facets could not be SET from the web UI, the facets
endpoint had no in-app consumer, and the registry's libraryTypes hints
rendered nowhere. This phase closes all three.
- ONE UNION-ALL sqlc query (FacetCountsUnderFilters) computes all 14
listable facets in a single round trip: each arm omits its OWN
facet's filter block and carries every other registered facet's
block (copied mechanically from SearchMediaItemsUnified) — classic
faceted semantics: with filters active each facet counts under the
others; with none, blocks are neutral and the counts are the v1
global-per-library ones. One query serves both cases. The 14
per-column Batch C queries are deleted.
- Endpoint: no facet param → the new all-facets shape
({facets: {param: [{value, display?, count}]}}), filters bound via
the shared ParseMediaSearchParams, library_id optional (the
All-Libraries view, family lens still enforced); facet=X keeps the
pinned single-facet shape (now routed through the same query with
neutral filters).
- SSR partial GET /media-items/facet-panel: templ renders the drawer's
value lists — registry order, Batch D library-type scoping
(FacetsForLibraryType: manga_type on manga, reading_direction on
manga+comics, narrator on audiobooks; All view = union), prettified
genre labels + language display names, active values marked. Family
lens: out-of-set explicit library_id is 403.
- Drawer wiring: the panel fetches on drawer open and after every
apply with the CURRENT filter query string; a value click toggles
that facet's comma-joined filter (hidden input where the drawer has
no field — same as deep links) and re-applies the form; chips and
URL stay in sync. Values are server-rendered (rendering doctrine);
bookshelf.ts owns only the click/fetch orchestration.
- prettifyGenre moves to internal/utils as the canonical
implementation (services' display enrichment shares it; templ wraps
it; the TS port in bookshelf.ts is unchanged).
- Drift guard extended: FacetCountsUnderFiltersParams must carry every
registry facet's param, the bools as pgtype.Bool.
Tests: under-filter exactness (own-block omission, OR-within,
AND-across), family lens (hidden values absent; 403 on both shapes),
all-shape + pinned-shape coexistence, partial rendering with Batch D
scoping + active marking + 403. Full container suite green except the
documented pre-existing TestAdminCreateUser red.
|
||
|
John O'Keefe
|
5245f292ca |
feat(scanner): quiet scans, enqueue coalescing, watcher probe, sibling cache
Part 2 of the faceted plan, Phase 5 (steps 5-6): the scan that used to print ~4 lines per unchanged file now prints ONE summary line, and a poll tick landing mid-scan no longer stacks a second full scan. - Quiet scans (§4): the walk loop is silent — inserts, changes, and errors log inside processMediaFile; the duplicate 'Successfully processed' branch (both branches printed it) and the per-file 'processing / already exists / same size, skipping' chatter are gone. Summary carries the trigger and a changed counter: 'Scan of <folders> (trigger: poll): N files, N changed, N new, Xs, N errors'. Triggers threaded through every enqueue site: poll (startBackupScan), watcher (dirty-dirs flush), admin (scan API), initial (startup scan). - Enqueue coalescing (§5): Worker.HasActiveScanForLibrary (JobResult gains additive job_type so only SCAN jobs coalesce — an import sharing the library id never blocks one). enqueueLibraryScan skips silently; the admin scan API answers 409 with the active job id. Enforces the one-scan-per-library invariant the bulk scan-state map leans on. - Watcher capability probe (§4): ONE Add on a root folder; failure disables the watcher with a single startup line (replaces the per-directory warning spam on inotify-broken deployments). The watcher stays an optional latency reducer over scheduled scans. - Per-directory sibling cache (§6): hasSiblingBookFileCached caches the ReadDir per directory for one scan — loose-image manga re-stats thousands of images sharing a handful of directories. Tests: TestHasActiveScanForLibrary (unit — running/pending coalesce, terminal jobs don't, non-scan jobs don't, empty library never, nil worker safe). Full scanner suite green in the container. |
||
|
John O'Keefe
|
7e586df48a |
feat(facets): facet registry, Batch A server params, genre model v2, facets endpoint
Phase 1 + backend of Phase 2 of the faceted-navigation plan. One
declarative facet registry (services/facet_registry.go) drives the
bookshelf filter system; a drift-guard unit test reflects over the
generated SearchMediaItemsUnifiedParams so a registry entry and its
sqlc.narg block can never disagree silently.
- Nine new filter params on SearchMediaItemsUnified: story_arc,
imprint, edition (scalar-fuzzy), age_rating, format_group,
manga_type, reading_direction (exact), explicit, abridged
(presence-bound bools). Comma = OR-within multi-select on every
text facet; AND-across-facets unchanged.
- Genre model v2 (user rule: a subgenre is not a genre, it's a tag):
genre_lexicon.go rebuilt as shelf set + subgenre->shelf map;
pickSmartGenre picks direct shelf > mapped subgenre > Fiction floor
> empty (unknown-only stays tags-only). Type-words (Manga, Comics,
Graphic Novel) join the demote list. Both 'adventure' and 'action &
adventure' are shelves — provider data uses both and the verified
traces pin them.
- Language filter UX: comma elements normalize through the ingest
language map, so display names ('English') match rows stored as ISO
codes ('en'). The language autocomplete now serves display names.
- NEW GET /api/media-items/facets?facet=X&library_id=... — registry-
driven value+count listing (14 static per-column queries, family-lens
join); language values carry display names; bool/range facets 400.
- Family-lens gates: an explicit hidden-library library_id is now a
403 on the search endpoints (was a silent empty set) — JSON branch
and HTMX grid partial both.
- handlers.ParseMediaSearchParams is the SINGLE param parser for the
search JSON endpoint and the HTMX grid partial; the HTML branch
previously dropped publisher_filter and narrator_filter silently.
Pins: docs/contract_pins.md changelog entries (additive request
params; new facets endpoint). Tests: facet_registry_test.go (shape +
SQL drift guard + library-type union), series_parsing_test.go genre-v2
cases, facets_integration_test.go (param filtering e2e, counts,
language display, 403s) — container runner.
|
||
|
John O'Keefe
|
7537063cb7 |
feat(filters+nav): author navigation end-to-end — arrays on every media response, deep-linkable bookshelf filters, clickable authors
Completes the A2c acceptance scenario: click an author on a book detail page -> land on the bookshelf filtered to everything that author wrote (including their share of co-authored books). - Media detail + unscoped list resolved maps expose subtitle, explicit, authors, publishers (the library-scoped raw branch and the search endpoint already did — raw rows follow the SELECT). - SearchMediaItemsUnified SELECT carries the new columns; the search endpoint serializes the row directly, so clients get them there too. - book_detail.templ renders each author as a link to /bookshelf?author_filter=<name>. - Bookshelf honors filter params in the URL on load (author_filter, series_filter, genre_filter, publisher_filter, tags_filter, narrator_filter, language_filter): pre-fills the form, applies it to the first grid load. Comma-joined values are multi-select via the server-side string_to_array split. Filtered views are shareable. Pins widened: search rows (+7 keys), resolved keys (+4); changelog entries in docs/contract_pins.md. Android follow-up plan written to ~/.opencode/plan/bookhoard-app-v2-compat.md. |
||
|
John O'Keefe
|
f94e7472b4 |
feat(filters+ui): per-element author/publisher filters, playlist toggles, Up Next API, reorder UI, dashboard badges
Phases 4 + 5 of the metadata overhaul (plan §4, §5) plus §3 handler wiring. Filters (§4.1): author_filter/publisher_filter now match per-element against the authors[]/publishers[] arrays — a co-authored book matches any one of its authors — with comma-joined params acting as OR-within-facet multi-select (string_to_array + cross unnest). Legacy single-value params unchanged on the wire; series/genre/language filters stay on their varchar columns; narrator filter already per-element. Collections API (§3.3-3.5): collection detail carries viewer + hide_finished into the listing query and exposes up_next (visible- library-enforced reading-queue head); PUT /api/collections/:id/items/order (owner-only, full-order rewrite); UpdateCollectionRequest gains hide_finished/show_up_next ([] string last-value-wins for the hidden-false-sibling checkbox pattern) merged server-side OVER stored view_settings so imported_from_series provenance survives edits; EditCollectionModal renders the toggles from view_settings. Dashboard (§5.5): sections JSON carries is_playlist/complete (dashboard.ts is the AGENTS-documented JSON-render exception; first paint stays templ). Fully-read playlists render '✓ Playlist complete' instead of an empty rail; imported playlists get a 📖 Playlist badge. Frontend: book detail shows subtitle + full authors + Explicit badge; edit modal gains subtitle, comma-separated authors/publishers (client- split to arrays exactly like narrators), step=any series number, explicit checkbox; collection book cards get playlist move ↑/↓ buttons persisting the full order via the reorder endpoint (templ partial re-render on failure — DOM order is client-local, API order is truth). AGENTS.md gotcha #3 amended per plan §3.7: user collections are dashboard sections by default; the show_on_dashboard premise is retired. Verified: full affected integration sweep green (pins widened for up_next/is_playlist/complete — changelog entries in docs/contract_pins.md); unit suite green; go build + templ + vite green. |
||
|
John O'Keefe
|
1dbb0ce406 |
feat(collections): playlist auto-creation, position ordering, hide_finished, Up Next, dashboard eligibility
Phase 2b + 3 of the metadata overhaul (plan §2.4, §3.1-3.7). Auto-playlist collections (§2.4): the scanner turns obviously-playlist sidecar series (explicit '(Reading Order)'/chronological/publication/ in-order/timeline markers only — precision over recall) into curated static collections: upsert by view_settings.imported_from_series then name (rename-safe), position set ONLY on insert (clean integers; 1.5/1a stay unpositioned rather than guessing), membership add-if-missing, never removed or re-ordered by rescans, provenance in description + view_settings, show_on_dashboard true (dashboard-eligible by design), 'scanner.auto_playlist_collections' setting (default on, Settings Registry + schema seed). Ordering (§3.1-3.2): collection listings order position ASC NULLS LAST then added_at ASC (was added_at DESC — walked playlists backwards); PUT /api/collections/:id/items/order applies full-order rewrites (owner-only, service-gated). hide_finished + Up Next (§3.3-3.4): GetCollectionItemsForDashboard gains viewer_user_id + hide_finished (LEFT JOIN reading_progress, COALESCE percentage < 0.999) and viewer_percentage rides the row for card graying; GetCollectionUpNext serves the collection's reading-queue head with the family-lens visible-library array enforced. Dashboard (§3.5): user collections are dashboard-ELIGIBLE BY DEFAULT — the show_on_dashboard gate (never writable for user rows since the column shipped) is gone from GetUserCollectionsForDashboard; hiding stays dashboard-prefs (hidden_collections) territory; hide_finished filtered-everything now emits a Complete section instead of silently vanishing; sections JSON carries is_playlist/complete. Series page ordering hardening (§3.6): GetSeriesBooks tiebreaks by title ASC — NULL/tied numbers previously returned heap (scan) order, which is what the user saw. Frontend custom-section-builder note: the builder has sent show_on_dashboard: true since its creation; the API never bound it. User-row visibility is now intentionally prefs-governed, so the field stays unbound by design (documented in code). Pins widened (docs/contract_pins.md): collection detail + up_next; dashboard sections + is_playlist/complete; new reorder endpoint recorded. Verified: TestPlaylistCollectionsEndToEnd (new, full e2e: sidecar series split, collection auto-creation with position 7, rescan idempotency, owner-only reorder, finished-filter + completion marker + Up Next drain) plus the full affected integration sweep green. |
||
|
John O'Keefe
|
646477afad |
feat(schema): subtitle/explicit columns, authors+publishers arrays, float series numbers
Phase 1 of the series/collections/metadata overhaul
(~/.opencode/plan/series-collections-metadata-overhaul.md).
Schema (in place, idempotent):
- media_items.series_number INTEGER -> DOUBLE PRECISION: real-world
ordinals are decimals (0.5, 1.5 novellas); INTEGER silently truncated
them (calibre 2.5 pinned as 2 in tests).
- + subtitle TEXT, explicit BOOLEAN (sidecar ingest; ABS explicit /
Audible isAdult).
- + authors/authors_search/publishers/publishers_search TEXT[] NOT NULL
(narrators A2a precedent). Arrays are write truth; author/publisher
varchars become trigger-regenerated display strings
(sync_author_publisher_display) so every varchar consumer (trgm
search, facets, API, OPDS/Kobo) keeps working unchanged.
- Row-level NULL normalization for the new arrays in
set_library_type_name (A2 lesson: Go nil -> explicit NULL -> 23502).
- Backfills ARRAY[author]/ARRAY[publisher] — deliberately no
comma-splitting ('Tolkien, J.R.R.' would corrupt; scanner kept only
authors[0] historically).
- collection_items.position INTEGER + (collection_id, position) index
for playlist ordering.
Go: sqlc regen; MediaMetadata/handler params gain Subtitle/Explicit/
Authors/Publishers (legacy singles still accepted, arrays win);
SeriesNumber int32 -> float64 end to end; metadata_overrides adapted
with legacy override keys ('author'/'publisher') preserved so existing
user overrides survive; bulk author edit writes the array.
Pins widened (docs/contract_pins.md): raw sqlc row key set + the six
new columns; changelog entry added.
Verified: unit suite green; integration sweep (bulk/smart collections/
scanner/stats/sync/queue/calibre/pins/audiobooks) green against the
shared dev DB after in-place migration. Pre-existing unrelated red:
TestOwnershipAndAdminTestSuite/TestAdminCreateUser timezone inheritance
fails identically at the pre-change commit (live shared state).
|
||
|
John O'Keefe
|
6216bd9bcf |
feat(audiobooks): multi-file books (folder-of-tracks + zip) + exhaustive metadata ingestion
Multi-file book ingest: a directory of ≥2 audio files (recursively —
disc-part folders like CD1/Disc 2 merge upward, siblings never merge)
ingests as ONE book whose file_path is the book-root directory; a zip
of audio tracks is its own book (file_path = the archive). Both write
the documented chapters abstraction {index,title,start,end} into
audio_metadata and add additive keys: layout ("folder"|"zip") and a
tracks[] manifest (per-track path, global-seconds start/end, duration,
size). Pre-grouping one-track strays under a book root are absorbed on
rescan (annotations cascade). Rescans take a stat-only fast path
(track count + total size + known durations unchanged → no re-probe —
a 1189-track book must not re-ffprobe every auto-scan); a zero-duration
book counts as changed so a ffprobe-equipped rescan fills durations in.
Folder-book roots count as present in the archive lifecycle pass, and
per-item rescan (RescanMediaItem) routes multi-file books through a
book-level refresh — the generic updater would read the root/zip as if
it were the book file and clobber format_group with "unknown".
Serving: GET /api/media-items/:id/chapters gains additive layout +
tracks keys (urls → the new endpoint). New GET
/api/media-items/:id/audio/:trackIndex streams one track — folder
tracks served from the real file (full Range via ServeContent), zip
tracks via a reopen-and-skip ReadSeeker over the entry (deflate has no
random access; acceptable at audio bitrates; stored entries stay
section readers). Both endpoints gate on library visibility (403 on
hidden libraries — family-lens rule). ServeFile and OPDS DownloadBook
refuse directory paths (folder books have no single-file download).
Web player (next commit) consumes the manifest; progress/bookmarks/
deep-links keep their global-seconds coordinates, so the server-side
progress model is untouched. sync learns FormatGroupAudiobook
(percentage passthrough in ConvertProgress); detectFormatType stops
reporting "unknown" for audio.
Metadata ingestion (Batch B) — read everything available, degrade
gracefully, documented precedence sidecars → folder structure →
embedded tags → filename: ffprobe tags are now a case-insensitive
lookup map (freeform keys pass through verbatim); composer fills
narrators (Audible's own convention, kept as ONE string — no reliable
multi-narrator delimiter); grouping + Audible freeform atoms
(SERIES/SERIES-PART/SUBTITLE/ASIN/ABRIDGED) fill series/index/
subtitle-join/ASIN/abridged (explicit sidecar abridged-false is
guarded); date/genre fall back to tags; album/album_artist/artist
fill title/author. Genre persists at create for all formats.
Embedded cover art (mp3 APIC, m4b covr, flac picture) extracts via one
optional ffmpeg call as a "<book>.cover.jpg" sibling — sidecar covers
win. CUE sheets (FILE-matched, INDEX 01 at 75fps) are the lowest-
priority chapter source under sidecars and embedded m4b chapters. A
lone m3u/m3u8 in a book root imposes playlist ordering + #EXTINF
names; (disc, track) tags reorder tracks when coverage is complete.
extractMetadata's default branch no longer pre-fills the filename as
title (tags must outrank it) — final filename fallbacks now exist on
BOTH the create and rescan paths (TestRescanKeepsFilenameTitle pins
that rescans cannot blank a title); tagless zip books take the zip's
own name, not their parent folder's.
Accepted containers widen to .m4b .m4a .mp3 .flac .zip .opus .ogg .oga
.aac .wav with real media types registered (audio/ogg per RFC 7845 for
Opus-in-Ogg) — all browser-playable; .wma deliberately excluded.
Tests: unit tests for CUE parsing, playlist hints, tag fallbacks
(synthetic ffprobe reports — ffmpeg cannot write Audible freeform
atoms), ordinal parsing, tag ordering, natural sort, grouping rules;
integration suite (11 tests) over the live dev DB: folder/zip ingest +
idempotency, stray absorption, chapters manifest, per-track streaming
with Range, hidden-library 403s, embedded covers + sidecar-wins, CUE
chapters, embedded m4b chapters, playlist ordering, tag fallbacks, all
five new containers with content-type assertions, rescan title pin.
Fixture media is real committed-format binary content living untracked
under internal/services/testdata/ (gitignored by policy; recipes in
docs/contributing/development.md → Media fixtures). Test image now
ships ffmpeg to match the runtime's audio toolchain.
|