Commit Graph
2118 Commits
Author SHA1 Message Date
John O'Keefe 0741e98ea5 feat(admin): all-podcasts view — disk-first maintenance for every feed
§0 item 8 (Phase-4 item 1 pulled forward, user directive: maintenance
tooling ships WITH the features it maintains):

- /admin/podcasts (AdminMiddleware): every feed from EVERY user via the
  RLS admin escape (first-admin identity, poller/fetcher contract) —
  feed + host, library, subscriber count/names, downloaded/total, DISK
  USAGE, last fetch. Sorted disk DESC ('what's eating the disk'),
  totals header (feeds · episodes · bytes on disk).
- Per-feed actions: Refresh · Delete files (rows → pending, disk freed;
  confirms spell out the re-materialization contract) · Remove feed
  (rows + files for everyone — the same admin delete as the detail
  page). All confirms ride the global confirm modal.
- New sqlc: ListAllPodcastFeedsAdmin (correlated aggregates),
  ResetPodcastEpisodeFileDeleted / ResetPodcastFeedFilesDeleted —
  downloaded rows reset to pending with download-derived facts cleared
  (audio_metadata, chapter_count, cover, size, mime); file_path KEPT
  (fetch-maintained layout — a re-download lands on the same path).
  file deletion via the existing DeleteFeedFiles file-removal pattern
  (episode files + .cover.jpg siblings).
- Poll upsert never touches download_state, so deleted files STAY
  deleted — a feed refresh re-lists metadata but re-materializes
  nothing (pinned by TestAdminDeleteEpisodeFiles).
- Per-episode admin delete-file: POST /feeds/:id/episodes/:eid/
  delete-file (admin) — one episode's file gone, row pending, siblings
  untouched; HX swaps the episode list. (Found while testing: the
  handler first read c.Param("id") — the FEED's uuid — instead of
  :eid; the 404 'episode not found' trail led straight to it.)
- Tests: admin sees other users' feeds + subscriber names; regular
  users 403 on page AND actions; single + feed-level deletion reset
  rows/clear facts/delete files; refresh leaves pending rows pending.
- Admin nav gains 'Podcasts' under Administration.

Wrong-gotcha note for future agents: parseUUID on c.Param('id') inside
a /feeds/:id/episodes/:eid route resolves the FEED, not the episode.
2026-10-30 13:54:37 -04:00
John O'Keefe b293741eb1 feat(reader): podcast episode prev/next on the transport row's outer buttons
User design (round 3): one set of big buttons does the big navigation.
The outer prev/next step buttons were chapter-only even on podcasts,
while episode nav hid as small icons in the top-left cluster. Now:

- Podcasts (sequence context present): the outer buttons navigate
  EPISODES via goToPrev/goToNextEpisode (progress pushed before the
  hop); disabled ghost buttons at the first/last episode keep the row
  shape. Embedded-chapter jumps stay on the Chapter pill's list.
- Books: unchanged chapter stepping.
- Keyboard (h/l shortcuts) and hardware media keys (MediaTrackPrev/
  Next) funnel through stepChapter, which is now sequence-aware —
  headset next-track = next episode on podcasts.
- The top-left episode icons are gone; the 'Ep N/M' counter stays.
- Help-modal labels say 'chapter, or episode on podcasts'.

Same icons both modes (extracted audioStepPrev/NextIcon partials);
books are untouched — no sequence context, byte-identical behavior.
2026-10-30 12:46:36 -04:00
John O'Keefe ace9e57d27 fix(ui): hx-confirm interception actually worked — htmx 2 detail shape
User report: Remove Feed still used the browser dialog. Headless-proven
root cause (tests/ui/confirm_probe.spec.ts): the htmx:confirm handler
read evt.detail.element, which htmx 2 does not provide (its detail is
{ elt, question, issueRequest, ... }) — the handler returned at
'if (!elt) return' BEFORE preventDefault, so every hx-confirm sitewide
fell through to htmx's native confirm. The acbcc61a migration only ever
worked for direct confirmDialog() callers; hx-confirm interception was
dead on arrival against htmx 2 and no test watched the JS side.

Fix: read detail.question (htmx's own closest-attribute resolution of
hx-confirm); return without preventing default when absent. Probe
spec pins: click Remove Feed → #confirm-modal opens with the message,
cancel hides it, no dialog event fires.

Companion fixes in the same user-test round (one commit, one defect
family — templ boolean attributes + confirm-modal placement):
- Add-modal Download order never revealed: #podcast-download-all was
  nested in a wrapper div, so ':checked ~ .podcast-add-order' had no
  sibling relationship. Input, label, and order row are now siblings.
- Order/auto-download dropdowns always DISPLAYED the last option
  ('Newest first'/'Newest') whatever was saved: templ selected={ bool }
  emits selected="false" — presence selects, so every option was
  selected and the browser honored the last. Swept to selected?={...}
  across podcast_detail (14), book_detail_modals (10, incl. two
  always-checked checkboxes), collection_modal (2, same). Saves were
  correct all along; display only.
- Global confirm modal moved from SidebarSignIn (signed-out only!) into
  Header — signed-in pages had no #confirm-modal, so confirmDialog
  degraded to window.confirm even with a working interceptor.

Facet bar scale (user question): values now carry feed counts, sort
frequency-first, cap at 6 chips per group with a native <details>
'+N more' expander.

Probes: tests/ui/confirm_probe.spec.ts + add_order_probe.spec.ts
(live-server, BH_FEED-gated) — both pass against the dev instance.
2026-10-30 12:24:16 -04:00
John O'Keefe 2fd0828c06 fix(podcasts+ui): user-test round 3 — five browser-reported defects
1. Add-modal 'Download order' never revealed: the CSS sibling selector
   (#podcast-download-all:checked ~ .podcast-add-order) can't fire —
   the checkbox was nested inside a wrapper div, so the select row was
   no SIBLING of the input. Restructured: input, label, and order row
   are now siblings under one flex-wrap wrapper.

2. Order + auto-download dropdowns always displayed the LAST option
   ('Newest first'/'Newest') regardless of the saved value: templ's
   selected={ boolExpr } compiles to selected="false" — and the mere
   PRESENCE of the attribute selects the option, so every option was
   selected and the browser honored the last. Sweep to the boolean-
   attribute form selected?={...} (omits on false) across
   podcast_detail (14), book_detail_modals (10: manga type/reading
   direction selects + abridged/explicit checkboxes — same defect,
   always-checked), collection_modal (2: hide-finished/show-up-next
   always-checked). Values were SAVING correctly all along; this was
   pure display. audio_reader/collections :selected/:checked are
   Alpine bindings — unaffected.

3. Remove Feed (and every hx-confirm) still showed the browser dialog
   for signed-in users: the global confirm modal rendered at the end
   of SidebarSignIn — signed-IN pages have no SidebarSignIn, so no
   #confirm-modal existed and confirmDialog() degraded to
   window.confirm. Moved the modal into Header proper (rendered on
   every page, after </aside>).

4. Facet bar at scale: values now carry feed COUNTS, sort by frequency
   DESC then alphabetically, cap at 6 chips per group flat, and the
   rest collapse into a native <details> '+N more' expander — hundreds
   of hosts/tags degrade gracefully instead of a wall of chips.

Verified live on the dev instance (order select reflects the saved
override, override row persists, reveal fires, modal renders on
signed-in pages). Podcast + logout + media-pin suites green; vitest
167 green.
2026-10-30 12:03:44 -04:00
John O'Keefe 15f41e6fd4 test(pins): whitelist download_queue_pos on raw media rows (additive widening)
The contract pin caught the §0 item 4 column surfacing on media-list
raw sqlc rows — its job. Widened additively per doctrine: rawRowKeys +
docs/contract_pins.md changelog entry. NULL for scan rows and
never-queued episodes; nothing removed or renamed.
2026-10-30 10:40:34 -04:00
John O'Keefe 0da8e0cdf9 test(settings): cover the new float setting type in the defaults audit
TestSettingDefaults enumerates registry types; 'float' hit the unknown-
type branch. Added float validation (default parses, within [min,max])
mirroring the int case.
2026-10-30 10:31:02 -04:00
John O'Keefe b574df346a chore(podcasts): drop the dead QueueFeedDownloads wrapper
All callers moved to QueueFeedDownloadsAs in the item-5 commit; the
no-override shim had no remaining callers.
2026-10-30 10:29:45 -04:00
John O'Keefe f0f1eb6e8f feat(podcasts): hide played/downloaded toggle + episode text filter
Pocket Casts steal pulled forward from Phase 4 (§0 item 6): the episode
list gains a client-side 'Filter episodes…' box and a 'Hide
played/downloaded' toggle. Pure presentation — rows carry templ-rendered
data-* attributes (downloaded, listen-pct, search text); the filter
toggles DOM visibility with the 99.5% finishedThreshold. The toolbar
lives OUTSIDE #episode-list so its state survives the list's polling
swaps, and an htmx:afterSwap listener re-applies the active filter to
freshly swapped rows. No query change, no server round trip, no new JS
rendering surface (Alpine component method, CSP-safe).
2026-10-30 10:28:31 -04:00
John O'Keefe 5c545534c2 feat(podcasts): Download All queues in the LISTENING order + add-modal order dropdown
Download All previously queued in feed-native order — 'getting the
last chapter first doesn't help me'. Now the queue follows the feed's
effective listening order (§0 item 5b): QueueFeedDownloadsAs reads the
CALLER's order override and runs episodes through services.OrderEpisodes
(same helper as the sequence/reader/page surfaces — the consolidation
from item 5a landed with the reverse-order commit). Queue tickets
(SEQUENCE-stamped, item 4) make that enqueue order the claim order, so
chapters land in the order you'll actually listen.

Add modal (§0 item 5c): with 'Download all episodes now' checked, a
'Download order' select appears (pure-CSS :checked sibling reveal — no
JS island, CSP-clean). Its choice is the feed's INITIAL order override
(reader_settings upsert; 'feed' = no row) AND the queue order for the
initial batch. Per-book by design: the override IS the download order.

Test TestPodcastAddModalDownloadOrder: add with download_all+reverse →
the override row exists AND persisted tickets run position 3→1. Tickets
are asserted (not live queued rows) because the shared dev server's
worker legitimately races this test's queue — tickets survive every
state flip.
2026-10-30 10:26:00 -04:00
John O'Keefe 0f3c4a8cec feat(podcasts): concurrent download pool — fair share, tickets, admin tunables
Replaces the sequential worker with a download-manager-style pool
(§0 item 4). Model: the user's connection is the limit; semantics are
fair-share ACROSS feeds, not goroutine-per-book (a global FIFO would
make Book B wait behind Book A's 100 files).

- Atomic claim: GetNextQueuedEpisode + SetPodcastEpisodeDownloading
  was a read-then-write race under concurrency; ClaimQueuedPodcastEpisode
  converts queued→downloading in ONE guarded statement (two claimants,
  one winner — the loser gets ErrNoRows). SetPodcastEpisodeDownloading
  retired.
- Queue tickets: download_queue_pos BIGINT fed by a Postgres SEQUENCE
  stamped at enqueue — the enqueue loop's ordering IS the claim order
  (the foundation §0 item 5 builds on; previously all rows in a batch
  shared one updated_at, so within-feed order was arbitrary).
- Pool: up to podcast_download_concurrency (default 4, live range 1–16)
  claim-loop goroutines per drain round; the setting re-reads every
  round so admin changes take effect without restart.
- Fair share: claim prefers the feed least-recently-served
  (pickFeedToServe; never-served first), earliest ticket within a feed —
  books proceed in parallel while chapters land in order (start
  listening while the rest download).
- Bandwidth: shared token bucket across all workers
  (podcast_download_speed_mbps, default 0 = unlimited); take() reserves
  bandwidth and callers sleep outside the lock.
- Settings: new float setting type end to end (registry defaults +
  clamping, save validation, admin UI number-with-step input) + the two
  schema seeds; worker wires the registry via SetSettings (tests run on
  registered defaults).
- Tests: unit coverage for the bucket reservation math + fair-share
  pick; integration TestPodcastDownloadPool pins ticket order = enqueue
  order, claim exclusivity, and a full 3-episode pool drain.

Dev-DB note: the column/sequence/settings were applied live (idempotent
DDL, same statements the schema runs at next boot) since the running
container predates them.
2026-10-30 10:12:43 -04:00
John O'Keefe e3675536aa feat(podcasts): reverse order — position DESC, date-agnostic; single order helper
The 'Oldest first' order sorts by PUB DATE, which breaks on Scribl's
migrated podiobooks feeds (messy/all-equal dates). New order value
'reverse' = feed position DESC with a date-DESC tiebreak — dates are
not involved, so nothing can break.

- services.OrderEpisodes: THE single order source of truth (§0 items
  3+5a). The four surfaces (sequence JSON, reader chain context, SSR
  page list) previously carried three near-identical handwritten
  comparators — all now route through one generic helper. The SSR page
  list gains a SortDate key (lexical 2006-01-02) so the pub-date
  tiebreak is actually chronological; the old comparator compared
  'Jan 2, 2006' display strings alphabetically (latent drift bug).
- Dropdown becomes four options: Feed order / Feed order (reversed) /
  Oldest first (pub date) / Newest first (pub date).
- Serial suggestion button renamed 'Reverse feed order' (sets reverse,
  not oldest) and its response out-of-band-swaps the order select
  (PodcastOrderSelectOOB — same mechanism as Download All's policy
  select), so the dropdown mirrors the server state immediately.
- Validation: oneof=native reverse oldest newest everywhere.
- Tests: reverse binds via the form wire; messy-dates sequence
  returns 3,2,1; existing reorder-safety stays green.
2026-10-30 09:15:03 -04:00
John O'Keefe 7c7c08748e fix(podcasts): download_all binds on every wire — FlexBool + flip log
Reproduced the §0 item 2 suspect: echo v5 bool-binds form values via
strconv.ParseBool, which accepts "true"/"1" but REJECTS the
browser-native <input type=checkbox> value "on" — and a bind failure
kills the WHOLE AddFeed request (400 "invalid request"), not just
the checkbox. The add modal posts urlencoded (htmx), so any
checkbox-serialization that emits "on" made Download All silently
unusable.

Fix: FlexBool (encoding.TextUnmarshaler-style UnmarshalParam) binds
download_all permissively on every wire — "true"/"on"/"yes"/"checked"
=> true, explicit false/0/off => false, ABSENCE (unchecked box on any
wire) => false; JSON bool binding is native and unaffected. Tolerates
the exact wires htmx and browsers emit.

AddFeed's subscription flip now logs "[PODCAST] flip ... <feed> —
user" so the live flip is provable in app logs (user suspected it
never fired).

Integration test TestPodcastDownloadAllFormEncoding pins the urlencoded
wire (what htmx actually sends): true + on both flip the caller's
policy to 'all', an unchecked add leaves the default.
2026-10-30 09:06:41 -04:00
John O'Keefe cdf81f6cf4 fix(podcasts): page-load polls during downloads; indeterminate bar slides
A fresh detail-page load while downloads were active rendered the
NON-polling episode container (PollingEpisodeList was only reachable
via the /episodes partial route after Refresh), so the page froze
until the user refreshed manually. PodcastDetailPage now picks the
variant the same way the route does — HasActiveDownloads selects the
every-2s poll container.

Also: the indeterminate progress bar (feeds without Content-Length,
e.g. Scribl) used animate-pulse, which fades in place and reads as
frozen; replaced with a sliding shimmer keyframe
(.podcast-progress-indeterminate, 33% sliver, translating -100% ->
300%) in input.css. The live MB counter stays.

§0 item 1.
2026-10-30 08:53:35 -04:00
John O'Keefe 7ea84043fa fix(auth): logout actually signs out — clear the HttpOnly session cookie
Release / build-and-push (push) Successful in 3m9s
Logout wiped localStorage but never cleared the token cookie (HttpOnly —
only a server response can) and never called the logout API, leaving the
refresh token live for 30 days and every cookie-authenticated SSR page
accessible: the fake-signout bug. Reproduced on prod and dev; unrelated
to RLS (it sits behind authentication) and long-standing.

- /api/auth/logout now ALWAYS clears the session cookie (Set-Cookie
  empty + Max-Age=-1, HttpOnly/Path/SameSite mirroring login) — even a
  malformed body must sign the browser out; refresh-token revocation
  unchanged.
- The header logout button (and the rules-page copy) now POST to the
  API with the stored refresh token BEFORE clearing local state and
  redirecting — best effort: local cleanup never blocks on API failure.
- toast 401 handler fixed: it cleared 'refreshToken' but storage uses
  'refresh_token' — the real key.
- Regression test TestLogoutClearsSession: real login → logout →
  response deletes the cookie; logged-out refresh token can't mint
  access tokens; body-less logout still clears.

Also carries regenerated artifacts from the confirm-modal and
episode-list work (header_templ/podcast_detail_templ/style.css).
v2.3.2
2026-10-29 22:01:32 -04:00
John O'Keefe 253d8d0e8e feat(podcasts+ops): Apple launcher-scheme feeds + visible build stamp
Release / build-and-push (push) Successful in 3m29s
Apple scheme normalization (follow-up to the Scribl discussion): Add
Feed accepts the launcher spellings podcast pages ship next to plain
RSS — itpc://, podcasts://, feed:// — and rewrites them to the https
feed URL they wrap (scheme match case-insensitive). http stays and the
guard still rejects it. Unit-pinned.

Build stamp (stale-container drift guard): the 2026-10-29 podcast
session lost an hour to a container running a build that predated the
feature. version.Commit rides ldflags from a GIT_COMMIT build arg; the
dev compose passes git rev-parse --short HEAD via the Makefile; the
app logs 'Bookhoard <ver> (commit <hash>)' at boot and /api/health
reports it — the UI and the repo can no longer silently diverge.
v2.3.1
2026-10-29 20:12:39 -04:00
John O'Keefe acbcc61ae4 feat(ui): global confirm modal — browser dialogs retired sitewide (issues #5/#4/#7)
ONE styled modal answers every hx-confirm in the app: a body-level
htmx:confirm interceptor (web/src/confirm.ts, canonical htmx pattern)
preventDefaults and shows the dialog, issuing the request on Confirm.
Nine hx-confirm call sites upgrade without touching them. The modal
lives in the header partial so every page has it; confirmDialog() is
exported for JS call sites and degrades to the native dialog if the
header is absent (bare layouts).

Migrated the five window.confirm sites (admin.ts ×3, book-detail.ts ×2)
to the same promise API.

Podcast feed detail (issues #4/#7): the unsubscribe copy is
role-aware — non-admins get 'Unsubscribe from this feed?' with silent
retention (file state is admin knowledge); an admin who is the LAST
subscriber gets a three-choice modal instead: Unsubscribe (files kept,
the D6 default — sliding-window feeds make purge unrecoverable) /
Remove feed + files / Cancel. Admins with co-subscribers keep the
files-note confirm.
2026-10-29 20:12:30 -04:00
John O'Keefe 30766020c8 feat(podcasts): real-time episode list — byte progress bars, direct swaps (issues #1/#3)
WHY: downloads needed manual refreshes to see progress; the 204 +
HX-Trigger event indirection missed its listener, and the worker
buffered whole enclosures so no progress existed to show.

- Worker streams enclosures to disk through a counting loop (64KiB
  chunks, scheme/redirect/size guards preserved) with an in-memory
  per-episode progress snapshot (received/total; total from
  Content-Length, seeded by the feed's declared length). Deliberately
  transient — the DB row state remains the durable truth.
- Episode-list renderer seam: the router injects a closure into
  PodcastHandler (handlers can't import templates — MediaDetail cycle)
  so htmx download/retry/Download All/refresh/order/policy actions
  answer with the REFRESHED partial directly (200, hx-target=
  #episode-list) — no events to miss. Non-HX callers keep JSON; tests
  without a renderer keep 204+trigger.
- Polling tightens to 2s while downloads are active; downloading rows
  render a percentage bar + MB counter (quote-free inline style per
  templ gotcha #8) or an indeterminate bar when the server sends no
  length.
- Issue #3: Download All (button AND the add-feed checkbox) flips the
  CALLER's subscription policy to 'all' — per-user, never global (D6:
  files shared, policies not) — and the auto-download select
  out-of-band-swaps so the dropdown shows what the user is getting.
  They can turn it down afterwards; the flip is honest communication,
  not coercion (user-approved 2026-10-29).

PINs: Download All with HX answers 200 + episode-list + oob select and
flips newest → all; add-feed download_all sets policy all.
2026-10-29 20:12:19 -04:00
John O'Keefe ffc80f7ba7 fix(podcasts): page search hits episode titles/descriptions (issue #8)
WHY: the /podcasts search box filtered feed CARDS only (title, display
name, host, publisher, feed description). D14's episode search existed
only as the /api/podcasts/search endpoint and was never wired into the
page — searching 'Quarter' (present only in episode titles) cleared the
grid while 'Trader' (feed metadata) survived.

The page query now consults SearchMyPodcastEpisodes (subscription-scoped
via RLS — the caller's identity) when a query is present; a feed
surfaces when its OWN metadata OR any of its episodes matches, with an
'N matching episode(s)' badge on the card.

PIN: SSR test with a needle that exists only in an episode title — the
feed must surface with the match count.
2026-10-29 19:49:39 -04:00
John O'Keefe c753780817 fix(podcasts): order selector validation — bind the htmx form wire format (issue #2)
WHY: htmx sends non-GET requests urlencoded by default; the SetFeedOrder
request struct only carried a json tag, so the binder left Order empty
and 'required' fired — the exact error the order dropdown AND the
Serial-suggestion button produced (hx-vals JSON also arrives
form-encoded). One form tag fixes both controls.

Regression test drives the exact wire format: urlencoded PUT with
order=oldest/native → 200, empty/invalid → 400.
2026-10-29 19:45:46 -04:00
John O'Keefe 8d9fb0046c refactor(podcasts): drop the dead downloadRoot param on upsertEpisode
FilePath is library-relative (the pinned layout resolves at download
time via the library folders); the parameter never carried a value.
2026-10-29 18:22:09 -04:00
John O'Keefe 5c10bbb90e test(podcasts): reorder-safety pin — progress survives position renumbering (task 2.6)
Invariant 3: progress anchors on the episode row (guid identity +
chapter/offset), never the ordinal. Renumbering feed_position (the
fetcher's per-fetch renumber case) leaves the progress row untouched
and the sequence membership stable — downloaded-only with no gaps.
2026-10-29 18:17:51 -04:00
John O'Keefe 49b0b6eeb9 test(podcasts): integration suite + RLS auditor + contract widening (task 1.10)
cmd/server/tests/podcast_test.go — six suites, the shared-dev-DB
doctrine (test*-named libraries, cascading cleanup, no binary
fixtures):

- TestPodcastRLSSurfaces: THE AUDITOR (invariant 13) — non-subscriber
  404 on feed detail + SSR page (existence leak), episodes absent from
  generic media reads, admin escape, subscriber visibility, then the
  hidden-library flip hiding an already-subscribed feed (the feed
  policy's library-visibility join). Seeding runs as the FIRST ADMIN —
  the seed identity has neither subscription nor users row, so podcast
  INSERT..RETURNING (SELECT policy under FORCE) 42501s for it;
  production pods run as exactly that admin (invariant 12). All direct
  row probes read through the admin scope for the same reason.
- TestPodcastAddFeedFlow: member add + subscribe + metadata-only
  upsert; duplicate-URL dedupe (no second row); allow_member_feeds
  gate (member 403 / admin 201 / subscribe-still-allowed).
- TestPodcastShrunkenFeedRetention: feed purges an episode from the
  XML, both rows persist (upsert-without-delete pin, D14).
- TestPodcastEditSegregation: generic edit 400 + write-to-file refusal
  for episodes; control book edit unaffected.
- TestPodcastDownloadWorker (ffmpeg-generated tone at runtime): file
  lands at <download-root>/<feed-slug>/NNNN-…, audio_metadata probed;
  re-download of a downloaded episode never re-fetches (idempotency
  pin, enclosure-hit counted); dead enclosure → failed + persisted
  reason + metadata intact. Fetch/SSRF tests swap the client via
  SetPodcastHTTPClientForTests (the production client refuses
  loopback httptest servers by design — the real client gets its own
  negative test in services).
- TestPodcastScannerExclusion: SetFolders on a podcasts library root
  yields zero scanner folders (invariant 4 pin) — MediaScanner
  .FolderCount() accessor added.

Unit: internal/services/podcast_rss_test.go — namespace-aware parsing
fixture (three XML namespaces, fallback chains, hosts/guests),
duration spellings, RFC-822 variants, explicit variants, slug/path
layout, enclosure-extension selection, stripHTML, and the SSRF matrix
(private/loopback/link-local/multicast/mapped-v6 blocked; a loopback
httptest server PROVES the production client refuses it).

suites green: full integration run 161s / 0 failures; vitest 12 files
163 tests.

Contract widening: TestContractPin_MediaListBothBranches' raw-row
allowlist gains the 15 podcast episode columns (additive; documented in
docs/contract_pins.md — inert defaults on scan rows). Bruno: bruno/
podcasts/ mirrors the §8 endpoints (12 requests) + Bookhoard env vars.
tests/ui/COVERAGE.md rows 18-23: podcasts pages/lifecycle/RLS/chain —
Playwright sweep pending.
2026-10-29 18:09:35 -04:00
John O'Keefe 28420d56ed feat(podcasts): chain playback + Continue section — sequence mode in the web player (phase 2 core)
D7 chain-by-default, delivered with minimal player surgery:

- ReaderMetadata gains PodcastSequence: the neighboring DOWNLOADED
  episodes in the caller's effective order (D5 override applied,
  feed-native fallback; computed in the reader route for
  source='podcast_episode' items — RLS already scoped the reads; gaps
  never enter the chain, invariant 6). ?single=1 escapes chaining (the
  'play single episode' affordance).
- Audio reader renders the sequence into body[data-json]
  (AudioReaderConfigJSON — books produce the identical shape minus the
  sequence key, so the reader path is unchanged for them) and an
  'Ep N/M' indicator + prev/next episode controls in the header.
- audio.ts: the last-track 'ended' handler chains into the next
  episode's reader page (final progress push precedes navigation);
  back button exits episodes to their feed page. Books are untouched —
  no sequence context, zero behavior change. Day-one chaining is
  page-navigation based (progress persists; position restores).

Continue aggregation (2.5): GetPodcastContinueForUser (in-progress
episodes, most-recent-first, ≥95% excluded) feeds a Continue section
on /podcasts — sorted most recently listened, cards show 'Ep N · title'
— ahead of All Feeds. Cards prefer the continue line over the
description.
2026-10-29 16:25:44 -04:00
John O'Keefe 2235b00c59 feat(podcasts): web UI — /podcasts page, feed detail, download lifecycle (task 1.9)
Rendering ownership holds throughout: every piece of server data is
templ-rendered; htmx swaps partials; the only TS is the Add-Feed modal
toggle + error surfacing (pure client-local state, bundled via
web/src/podcasts.ts with automatic bundle-hash stamping).

Sidebar: /podcasts nav entry (D13, between Series and Collections),
gated by a ShowPodcastsNav flag resolved centrally in
getTemplateUserWithTheme (one indexed COUNT — same class of
per-request lookup as the live-user guard). New 'podcast' icon.

/podcasts: header + Add Feed modal (htmx post → HX-Redirect to the
feed page; inline error surfacing on failure), subscription-scoped
search + facet bar (host/tag/language — server-side GET filters, no
JS), grid cards with download counts / new-episodes / complete /
explicit badges. Empty states for no-library (admin creates it first)
and no-subscriptions.

Feed detail: hero (artwork, host/publisher, tags, fetch state), action
row — Download All, Refresh, ORDER SELECTOR (D5 override), serial
one-click oldest-first suggestion (never silent), auto-download policy
select (D6), Unsubscribe (files retained), admin Remove Feed — gap
banner (D8 count), and the EPISODE LIST with the full lifecycle inline
(D8/D19): pending greyed + Download button (every non-downloaded
episode has one), queued/downloading pulse, failed RED + reason text +
Retry (same endpoint). The list is a self-refreshing partial: re-fetch
on the podcast:changed HX-Trigger and polls every 4s ONLY while the
server says downloads are active (it renders the polling variant).

API endpoints grew HX awareness: 204 + HX-Trigger podcast:changed (or
HX-Redirect for add/unsubscribe/remove) — the sanctioned Go-handler
response channels; JSON callers keep JSON. AddFeed result exposes the
fetch-error text for the redirect.

Dates render in the requesting user's timezone (D17) via
utils.FormatTimeInTimezone; episodes play through the existing
/readers/{id} audio reader (episodes ARE audiobook items — no new
player code day one). CSP tripwire (templates/csp_source_test.go)
passes: no inline scripts, no on*= handlers, no hx-on/js: vals.
2026-10-29 16:17:55 -04:00
John O'Keefe 54d7d6e7a1 feat(podcasts): edit segregation — feed metadata is fetcher-owned (task 1.8)
D9/D10: users never edit podcast episode metadata. Three gates:

- Generic edit endpoint (UpdateMediaItem) rejects source=
  'podcast_episode' items with 400 — the next fetch refresh would
  clobber any edit anyway, and the fetcher is the sole metadata writer.
- Write-to-file (MetadataWriter.WriteMetadataToFile) refuses podcast
  episodes for the same reason (tag write-back for downloaded episodes
  is a named Phase-4 item needing its own iTunes-tag mapping).
- The book-detail page hides the admin Edit affordance for
  podcasts-type items (direct-URL visits), and the metadata-editor
  modal's field list has no podcast fields by construction (fixed
  templ field list) — episodes render read-only via /podcasts
  surfaces.
2026-10-29 16:05:03 -04:00
John O'Keefe 45b1152a15 feat(podcasts): API endpoints — add/list/detail, subscriptions, downloads, sequence, search (task 1.7)
The §8 surface, wired through main.go → router.Config →
/api/podcasts (JWT). Scoping is the engine's (D20): lookups outside the
caller's scope simply return no rows → 404; the only Go-level checks
are explicit-argument authorization (AddFeed's library +
allow_member_feeds gate) and the admin role on feed deletion.

Endpoints: POST/GET /feeds (add: fetch+parse+upsert+subscribe, optional
download_all; list: subscriptions + episode counts + facet values
computed in Go — feed counts are tens), GET /feeds/:id (detail with
download states + gap count), PUT/DELETE /subscription (the ONLY
user-writable podcast state, D9; unsubscribe retains files per D6),
POST /refresh, POST /episodes/:eid/download (+ episode_ids[] bulk and
eid='all' = Download All, exactly the non-downloaded set),
PUT /order (D5 per-user override in the reader_settings KV —
podcast_order:<feed_id>; 'native' deletes the row so default and
explicit choice are indistinguishable), GET /sequence (effective order
— override applied, native fallback — DOWNLOADED-only with the gap
list; this payload powers web chain mode and, per §15, the future
media3 queue), GET /search (subscription-scoped feed + episode search
covering un-downloaded and historical rows, D14), DELETE /feeds/:id
(admin; best-effort file removal, cascade deletes rows).

Gap definition (countGaps): an un-downloaded episode with a downloaded
episode AFTER it in list order — the D8 'middle episodes missing'
banner condition.
2026-10-29 16:03:35 -04:00
John O'Keefe a6723e6ac8 feat(podcasts): feed poller + podcast_poll_interval_seconds setting (task 1.6)
One ticker re-reading its interval from the settings registry each
cycle (live tunable — the cache stays honest because writes go through
the settings API path, AGENTS.md gotcha #2). Per tick: list feeds
(first-admin identity, invariant 12), skip is_complete feeds (back
off; explicit refresh still works), poll each (conditional GET via
stored validators), and apply the subscriber auto-download policy
union after each successful fetch so new episodes queue without
waiting a tick.

Schema/registry: podcast_poll_interval_seconds (default 3600s, floor
300s, category podcasts) — seeded with ON CONFLICT DO NOTHING per the
idempotency rule.
2026-10-29 16:03:22 -04:00
John O'Keefe 4288a86b2d feat(podcasts): download worker — state machine, idempotency, policy union (task 1.5)
One sequential worker (day one) over a DB-backed queue
(download_state='queued' IS the worklist — the episode list always
renders truth, D8/D19). State machine: pending → queued → downloading
→ downloaded | failed (+ download_error + download_attempts).

Idempotency: downloaded re-request is a no-op; a file already on disk
that probes cleanly re-marks the row without re-fetching
(crash-between-move-and-update reconciliation). Retry: manual path
resets attempts (human override), auto policies re-queue up to
maxAutoDownloadAttempts=5 then stay failed until a human retries.

Pipeline: enclosure fetch (same SSRF/size guards as the fetcher) →
.part temp → declared-size pre-check → MIME/size validation (empty /
over-cap / content-sniff refuses non-audio) → ffprobe reconcile +
embedded-cover extraction (scanner machinery reuse: runFFProbe,
extractEmbeddedAudioCover, AudioMetadataPayload) → atomic rename →
SUCCESS-ONLY row update (invariant 7: a failure never clears existing
episode data).

Policy union (D6): subscriber policies per feed — 'all' → everything
missing; 'newest N' → the N tail positions; 'off' contributes nothing;
auto-retry budget respected at queue time. DeleteFeedFiles for the
admin-only removal path (episode files + cover siblings + artwork).
2026-10-29 16:03:22 -04:00
John O'Keefe 85a9873a75 feat(podcasts): RSS fetcher — parse, SSRF guards, upsert-without-delete (task 1.4)
The fetcher is the SOLE writer of feed/episode metadata (D9). Three
files, no new deps (stdlib encoding/xml + net/http):

podcast_rss.go — RSS 2.0 + iTunes + Podcasting-2.0 structs. iTunes
fields are namespace-qualified in tags (local names collide with core
RSS: title/subtitle/author/summary/explicit/image/duration);
podcast:* fields match on local name (collision-free, tolerant of both
declared namespace URIs). Description/author fallback chains per §5;
itunes:duration parses seconds/HH:MM:SS/MM:SS/prose; pubDate accepts
the RFC-822/1123 spellings feeds actually emit.

podcast_http.go — the security boundary for user-supplied URLs:
https-only on every hop (CheckRedirect), private/loopback/link-local/
multicast IP ranges blocked at DIAL time (Dialer.Control sees the
RESOLVED address — DNS rebinding into 127.0.0.1 is refused at the
socket), 10-redirect cap, hard body caps (25MiB feeds / 20MiB artwork /
1GiB enclosures), no proxy.

podcast_service.go — fetch → parse → upsert. Episodes upsert
metadata-only keyed on (source_feed_id, episode_guid); rows for items
that vanish from the XML are NEVER deleted (D14 search covers
historical episodes). itunes:new-feed-url + permanent HTTP redirects
rewrite feed_url in place (row/subscriptions/episodes stay put);
podcast:guid dedupe reuses the canonical row instead of forking.
Artwork downloads feed-level under <feed-slug>/artwork.<ext>.
AddFeed authorizes ONLY on explicit arguments (library visible +
allow_member_feeds for non-admins — the D20 scoping model); detached
work runs under the first-admin identity via adminCtx (invariant 12).
Feed layout: <download-root>/<feed-slug>/<pos:04d>-<slug>.<ext> so
ServeFile/Range streaming resolve unchanged.
2026-10-29 16:03:22 -04:00
John O'Keefe 9a148c2344 feat(podcasts): generic-surface exclusions — podcasts never render as books (task 1.3)
§9 / D3 / D13 / D15 / D16: podcasts-type libraries are excluded from
every generic book surface. Under RLS these are pure UX/type filters
(engine already hides other users' episodes); their job is preventing
episode grids in surfaces designed for books. The discriminator is the
denormalized media_items.library_type_name (IS DISTINCT FROM 'podcasts'
— NULL rows predate the column and stay visible).

SQL (queries.sql, marked '-- podcasts exclusion (plan §9)'):
ListMediaItems/ListMediaItemsSorted (All Books) · SearchMediaItems +
SearchMediaItemsUnified (generic search) · FacetCountsUnderFilters (all
14 arms) + the six Search*Values facet-suggestion queries ·
GetDistinctSeries + Count (series pages) · GetLibraryItems (collections
picker) · dashboard system sections (Continue Reading / Recently Added
/ Recently Read / Not Started) · GetPopularBooks + GetTopBooksByMinutes
(book rankings must not rank episodes).

Go: statistics summary counts episodes' sessions into time-spent (D15
keeps that) but skips them in BooksActive/BooksFinished — an episode is
not a book. OPDS filters podcasts libraries via one helper at all 7
library-walk sites (D16).

Templ: LibrarySwitcher skips podcasts entries and TotalMediaCount
excludes them — presentation-layer filter so every switcher consumer
(dashboard, collections, custom-section builder) is correct at once
(D13); podcasts surface via /podcasts only.
2026-10-29 15:44:48 -04:00
John O'Keefe c0e6993273 feat(podcasts): scanner exclusion — podcasts libraries are feed-managed (task 1.2)
WHY: podcast download roots hold fetcher-written episode files; the FS
walker must never discover them (invariant 4) — a walk would double-
ingest episodes as scanner items and fight the fetcher's sole-writer
rule (D9). Every scan/watch path funnels through MediaScanner.SetFolders
(StartScanner, ScanLibrary jobs incl. scoped single-folder scans,
watch mode), so the filter lives there once: folders resolving to a
podcasts-type library are dropped before registration, with a log line.

Belt and braces: ScanLibrary's per-folder scope refuses purpose='podcast'
folders explicitly (400) — the panel-level path that bypasses type
resolution. AddLibraryFolder now derives purpose from the library type
('podcast' for podcasts libraries, enforced single download root;
'scan' default elsewhere), so folder purpose and library type cannot
diverge through the API.
2026-10-29 15:38:45 -04:00
John O'Keefe 0b7ad0f376 feat(podcasts): schema, RLS policy extension, sqlc queries — feed core foundations
Podcast integration task 1.1 (plan: podcast-integration.md §6/§8, D20).

WHY: podcasts are a new first-class media domain — RSS feeds users
subscribe to personally, with episodes as media_items rows so progress,
stats and the audio player work unchanged. Visibility is per-user
(strict: only your subscriptions), which after the RLS migration
(2026-10-29) belongs in the POLICIES, not Go helpers.

Schema (idempotent — schema.sql re-executes every boot):
- 'podcasts' library type seeded (ON CONFLICT DO NOTHING, ARRAY[]::TEXT[]
  cast required — bare ARRAY[] 42802s under ON_ERROR_STOP)
- podcast_feeds: per-library feed entity (UNIQUE(library_id, feed_url)),
  full RSS/iTunes/Podcasting-2.0 metadata inventory, conditional-GET
  validators, is_complete poll backoff
- podcast_subscriptions: the ONLY user-writable podcast state (display
  name, auto-download policy); deliberately UNGOVERNED — it IS the read
  policy's lookup table
- media_items episode columns: source/source_feed_id/episode_guid/
  feed_position/season/episode/enclosure_url/declared_duration/
  transcript_url/chapters_url/hosts/guests + download state machine
  (pending|queued|downloading|downloaded|failed + error + attempts),
  all partial-indexed so scan rows never touch them
- libraries.allow_member_feeds (first per-library knob); library_folders
  .purpose ('podcast' download roots, scanner-skipped)

RLS (inside the labeled policy block):
- p_media_items_visible gains the podcast-episode subscription predicate
  + admin escape (source IS DISTINCT FROM 'podcast_episode' guard keeps
  scan rows on the cheap path); unset identity still fails loud
- podcast_feeds governed identically: library-visibility join AND
  subscriber-or-admin SELECT; permissive write stand-ins (mandatory at
  ENABLE time; fetcher/worker declare first-admin, add-feed upsert uses
  ON CONFLICT DO UPDATE RETURNING so creators never need SELECT on
  foreign rows)
- Verified against dev DB: migration identity sees nothing, admin sees
  all, member sees feeds/episodes only post-subscribe (probed in a
  rolled-back tx — no rows persisted)

sqlc: 30 new queries (feed upsert/CRUD, subscriptions, episode
upsert-with-write-after-success-guard, download state transitions,
poller worklist, subscription-scoped search) — no visibility clauses
anywhere, per the D20 scoping model.
2026-10-29 15:36:11 -04:00
John O'Keefe 4ab4f9fa90 docs(readme): reconcile quickstart block post-inline-bootstrap
Release / build-and-push (push) Successful in 3m6s
The 12:21 sync (cf73eec2) made the README block a byte-copy of
docker-compose.yml as it stood before the inline db-init landed;
reconcile to the current file: initdb.d mount dropped, inline
hBcescaped bootstrap, PGPORT in db-init env (DB_PORT is a variable in
every connection path). Block verified byte-identical to
docker-compose.yml; no secrets in either version (placeholders only).
v2.3.0
2026-10-29 13:12:36 -04:00
John O'Keefe 304ea9599f feat(deploy): inline bootstrap into db-init — compose is the single source; DB_PORT everywhere
z_rls_role.sh is gone. The db-init service carries the bootstrap SQL
inline (the script's docker-exec/local-socket mode was its only reason
to exist as a file, and the no-clone deployment path could not use it:
docker bind-mounts of missing host paths auto-create empty DIRECTORIES,
so a compose-only install got '/z_rls_role.sh: no such file').

The inline form surfaced two silent bugs the script had masked:
- psql defaulted its DATABASE to the username (POSTGRES_DB is not a
  libpq variable) — the ownership loop ran against 'postgres' and
  no-opped on existing volumes. Fixed with PGDATABASE: bookhoard.
- PGPORT was never propagated: with a custom DB_PORT, db-init dialed
  5432 while Postgres listened on the custom port. Fixed with
  PGPORT: ${DB_PORT:-5432} — DB_PORT is now a variable in every
  connection path (db listener, host mappings, app, db-init).

Also removed the db service's initdb.d mount: the app executes the full
schema as the runtime-role owner on every boot incl. first (pg_trgm is
a trusted extension; the db owner can create it), so the bootstrap SQL
was its only content.

Verified: dev two-boot against the existing 40-table volume (owners
converged, 0 off-owner tables/functions), custom-port throwaway
(DB_PORT=15432: db-init + app both on the custom port, healthy),
default-port throwaway (fresh volume, healthy), full Go suite green.
Manual repair path: docker compose run --rm db-init.
2026-10-29 13:06:16 -04:00
John O'Keefe cf73eec2e4 docs(readme): sync quickstart compose block to docker-compose.yml verbatim
The 'copy this wholesale' docker-compose.yml block in the README had
drifted from the real make-of-record file: it hardcoded 5432/8765
(missing the DB_PORT/SERVER_PORT configurability from de8f71b2), lost
the loopback-only db publishing (b03ed9a2), carried its own hand-tailored
db-init service with inline SQL instead of the z_rls_role.sh script
version shipped in 313268a7, and still contained the vestigial
./uploads mount on the db service (dropped in the previous commit).

Replace the whole yaml block with a byte-for-byte copy of
docker-compose.yml so the quickstart can no longer drift — the
documented optional envs, loopback bind, db-init script path, and
env_file wiring all now match what actually ships. Verified by diffing
the fenced block against the file.
2026-10-29 12:21:07 -04:00
John O'Keefe ab4bddc784 fix(compose): drop vestigial uploads mount from db service
The db service mounted ./uploads:/app/uploads, which Postgres has never
used and nothing in the repo relies on: no SQL COPY TO/pg_dump path, no
doc/test reference to an uploads path inside the db container (the only
/app/uploads mentions in TEST_DATA.md are the app/test containers), and
the backup scan is app-side.

Forensics: the mount was introduced by accident in 6ebe9749 ("Simplify
docker-compose healthcheck configuration"), which incidentally REPLACED
the db service's volumes with the app service's uploads mount. The
follow-up 91456d11 ("restore essential database configuration")
restored postgres_data and the schema-init mount but nobody noticed the
stray uploads mount riding along, and it has survived every compose
refactor since.

The tests service in docker-compose.dev.yml keeps its own ./uploads
mount — that is the app-like test runner (TEST_UPLOAD_PATH=/app/uploads)
and is legitimate. The dev db inherits its volumes from the base file,
so it is fixed automatically.
2026-10-29 12:21:03 -04:00
John O'Keefe de26544d9d docs(readme): prod install without a repo clone — copyable compose
The Installation section assumed 'git clone the repo'. Self-hosted
operators only need three files. The install is now: mkdir + copy this
compose block + .env (three openssl secrets) + uploads dir + pull/up.

The compose is fully self-contained: the runtime-role bootstrap is
inlined into db-init's command (same SQL as
database/schema/z_rls_role.sh — keep in sync) — no ./database/schema
bind mount, because the app itself executes the schema as the runtime
role (owner) on first boot; with the role as DATABASE owner the trusted
pg_trgm extension needs no superuser.

Verified end to end in a throwaway project: fresh volume, pasted block,
.env with generated secrets, db-init converged role+ownership, app ran
the full schema and came up healthy — no repo checkout anywhere.

Compose-quoting lessons encoded (the reason for odd dollars): compose
interpolates '532016' to '$' before the container sees it, so the plpgsql
532016 tags in the inline SQL are written 532016532016, and PGUSER must be set
explicitly (POSTGRES_USER is not a libpq variable).
2026-10-29 12:05:56 -04:00
John O'Keefe efbe312f16 docs(agents): family-lens enforcement moves to RLS — one source
Approves and lands the doctrine rewrite from the verified rollout:
RLS on media_items IS the enforcement (policy joins library_visibility
on app.user_id, declared per transaction by ScopedDB); new
media-listing queries and endpoints need no visibility plumbing.
Documents the three identity bridge sites, background-job identities,
the FORCE/owner role and db-init provisioning, per-boot schema
idempotency, and the two auditors (TestRlsInvariant + tests/ui).
Item-scoped gates may answer 404 as well as 403 (contract_pins
2026-10-29). History records the convention-era leak the dashboard
section queries carried.
2026-10-29 12:01:37 -04:00
John O'Keefe d49a9e6ca2 fix(devices): device deletion detaches reading history instead of 500ing
Deleting any device whose stats rows still referenced it failed with
500: reading_history.device_id is a NO-ACTION FK by design (the rows
are real captured reading stats — Users reads them), so a blind DELETE
violated the constraint. Surfaced by the Android app's new logout
deregister call, but the admin Devices page Deletes hit the same stone.

Fix: DetachReadingHistoryFromDevice — NULL the device association on
the user's history rows before the device DELETE (single handler
pass). device_id is nullable; the stats survive, the device frees its
identifier. sqlc regenerated.
2026-10-29 11:57:20 -04:00
John O'Keefe 313268a7ae feat(deploy): db-init one-shot service — paste-and-go bootstrap for the runtime role
The role split previously required a one-time manual script run on
existing volumes (initdb.d only fires on fresh ones). This folds the
bootstrap into the compose file the way split-account deployments
standardly do it: a one-shot db-init service runs z_rls_role.sh
(idempotent, network mode via PGHOST=db + PGPASSWORD from DBPASS) on
every 'compose up', and the app gates on
db-init: condition: service_completed_successfully.

- Zero manual steps for fresh AND existing deployments: update compose,
  add DBSRVPASS to .env, 'compose pull && up -d'.
- Security posture improves: the superuser password is now held ONLY by
  the init service — the app container connects exclusively as the
  non-superuser runtime role.
- z_rls_role.sh gains dual connection modes (local socket for docker
  exec, network for the init service) with the DBPASS requirement
  enforced only in network mode.
- Verified on the dev stack: init runs and exits clean, app gates on it
  and comes up healthy, second 'up' re-converges idempotently, invariant
  test passes through the new dependency chain.

The initdb.d mount stays as-is (harmless double coverage on fresh
volumes; both paths are idempotent).
2026-10-29 11:17:42 -04:00
John O'Keefe 1457dc7289 docs(env): DBSRVPASS in .env.example — the runtime role password
.env.example is the template prod/new deployments copy; it was missing
the runtime role password the app now requires (DATABASE_USER
bookhoard_srv). Notes distinguish it from the bootstrap DBPASS and
point existing-volume operators at the one-time z_rls_role.sh run.
2026-10-29 11:02:41 -04:00
John O'Keefe f579696633 test(ui): Playwright RLS sweep — real-browser proof, both personas
Phase 5.5 (plan §7.5) first increment: the SSR/HTMX frontend is swept
in a real browser (official Playwright image, no host browsers) for
BOTH personas — testuser (admin) and rls-phone (restricted; Audiobooks
hidden from them via the visibility API).

Green checks: auth rejection + session establishment; admin dashboard
renders sections incl. the hidden-from-others library; restricted
dashboard renders real sections (Recently Added present — not a
vacuous landing-page pass) with ZERO hidden-library traces; search and
media payloads leak nothing for the restricted persona.

Harness lessons encoded: / is a landing page (dashboard lives at
/dashboard); the login endpoint rate-limits rapid attempts so sessions
are established once per persona and reused via storageState — a sweep
must not look like a brute-force run; getByText visibility assertions
are wrong for <option> text (library switcher) — body-textContent
assertions instead.

COVERAGE.md tracks the full 14-area sweep per the plan; the remaining
areas (reader, collections CRUD, admin matrix, upload roundtrip) are
enumerated and pending their sweep runs.
2026-10-29 10:25:08 -04:00
John O'Keefe 4d697f5fcf test(visibility): full suite green under RLS — sync paths, device-auth bridge, seed pool
Phase 5 (plan §7): the entire integration suite passes with the RLS
policies live. The suite was the triage enumerator the plan predicted:

- device_auth middleware: the THIRD identity bridge — KOReader/Kobo
  device-authenticated requests read media as the DEVICE OWNER
  (plan A9 called this path out; the flip was 404s on sync metadata).
- conflicts/analytics/progress handlers: detached
  context.Background() reads threaded to the request context (the
  requesting user's identity) — 12+ sites; resolve helpers take ctx.
- scanner: ScanFolders self-declares its configured admin identity
  (server maintenance, product-correct for any caller); generic job
  creation stamps the requesting admin.
- harness: setup.DBPool and the sync-test pool declare
  rls.SeedIdentity at SESSION scope via BeforeAcquire — the pool never
  serves a second identity, so no A3 bleed; setup.DB falls back to the
  seed identity (NewSeedDB) for ungoverned-context calls; filter tests
  use seedExec for raw governed-table statements.
- contract pin WIDENED (docs/contract_pins.md 2026-10-29): under RLS,
  item-scoped gates may answer 404 (row invisible — no existence leak)
  as well as 403 (Go-level checks on listings keep strict 403).
- TestWorker_JobStatusTracking — the pre-existing red from the Phase 0
  baseline — passes with the harness seed work; no product change.
2026-10-29 10:14:08 -04:00
John O'Keefe bb2cc26e9d feat(scanner): scanner entrypoints + generic jobs carry RLS identity; SeedDB harness fallback
The full suite enumerated the remaining triage class: tests drive the
scanner directly (ScanFolders(context.Background())) and read seeded
media via Queries with bare contexts — every governed read raised.

- MediaScanner.ScanFolders self-declares its configured admin
  (SetAdminID — set by main AND the test helpers): a scan is server
  maintenance, product-correct regardless of caller.
- ScopedDB gains a Fallback identity, exposed as NewSeedDB for the
  TEST HARNESS ONLY: setup.DB falls back to rls.SeedIdentity when the
  context carries none, so direct seeding/verification helpers work
  uniformly instead of per-site context threading. rls.SeedIdentity is
  the shared constant (no library_visibility rows => full visibility).
- generic job creation (jobs.go) stamps the requesting admin into the
  job Context, same as the scan endpoint.

The sync queue processor stays identity-free: sync_queue is not an
RLS-governed table.
2026-10-29 09:27:04 -04:00
John O'Keefe 385a5d9dba feat(scanner): background jobs declare acting identity for RLS
Phase 4 of the RLS plan: detached contexts (context.Background()) that
reach media_items now carry identities — with the policies live, a bare
context made governed reads raise (fresh connections: 42704 unrecognized
parameter; reused pool connections: 22P02 empty-string cast after the
LOCAL GUC reverted — the plan's A2 signature).

- scanner jobs (API-triggered, mtime-poll, initial scan): the job's
  Context now carries the acting admin's identity — the library owner,
  first-admin fallback (the job's UserID already encoded this trust
  decision; the Context just needed to carry it). performInitialScan
  uses the scanner's configured admin.
- websocket initial-sync read (GetUserMediaItemsForSync): runs as the
  connection's own user.
- hash backfill: runs as first admin (server self-heal over all media).

Verified on the dev stack: forced rescan of the 1199-file Audiobooks
library completed with 0 errors under RLS (previously every SHA lookup
and create raised); hash-backfill refusals gone. The sync queue
processor needs no identity — sync_queue is not an RLS-governed table.
Known remaining detached reads (conflicts/analytics handlers using
context.Background()) are HTTP-path triage, enumerated by the full
suite next.
2026-10-29 09:08:54 -04:00
John O'Keefe 922b92a8b6 feat(db): RLS library-visibility enforcement — invariant GREEN
The dashboard leak is closed at the database engine. The black-box
invariant (restricted user + hidden library => absent from every
media-returning surface) went RED->GREEN with this change set; clause-
protected surfaces never regressed, proving enforcement came from the
policies, not the test.

Phase 3 of plan ~/.opencode/plan/server-rls-library-visibility.md:
- schema.sql: idempotent RLS block — ENABLE + FORCE on media_items,
  SELECT policy mirroring COALESCE(is_visible,true) exactly, permissive
  per-command write stand-ins (FOR ALL would OR into SELECT).
- internal/rls + internal/database/scoped.go: ScopedDB declares the
  acting user per statement (LOCAL GUC). DESIGN CORRECTION during
  execution: the decorator never refuses — the ENGINE is the fail-loud
  layer (policies raise on governed tables; auth lookups and settings,
  ungoverned, simply work). A decorator refusal would have been a new
  convention gate — plan B thinking — and broke login (chicken-and-egg:
  credentials are checked before any identity exists).
- database/schema.go: the migration tx declares a synthetic identity —
  schema.sql's dedup DO block SELECTs media_items, which raises once
  policies are live (plan A11). Caught by the two-boot gate: boot 1
  created the policies after the block ran; boot 2 raised.
- router.go: RLS bridge at BOTH c.Set("user") sites (JWT + device
  auth) — echo-local c.Set never reaches the stdlib ctx the DB layer
  receives.
- search: the search service's raw DBTX path receives the ScopedDB
  itself, never the bare pool.
- harness: the server under test is wired exactly like main.go; the
  seed handle is also ScopedDB-wired but carries seedCtx() — a
  synthetic identity with no visibility rows — because sqlc :one
  inserts use RETURNING, which evaluates the SELECT policy even on
  INSERT. Regex UUID-scraping in the invariant replaced with a
  structured JSON decode.
- invariant test: RED -> GREEN across every surface incl. View All;
  403 pins and admin-scoping pins hold.
2026-10-29 08:56:05 -04:00
John O'Keefe 8c3d919aa0 wip(rls): engine-side fail-loud + harness seed identity [pin: work interrupted]
Design correction discovered by the auth chicken-and-egg: the
decorator-level refusal was re-created a convention gate (plan B
thinking). ScopedDB now NEVER refuses — it declares the GUC when the
context carries an identity and declares nothing otherwise; the ENGINE
is the fail-loud layer (policies raise on governed tables, ungoverned
tables like auth lookups simply work). rls.WithInternal removed as a
dead concept.

- search service's raw DBTX path now receives the ScopedDB itself
  (main.go + harness) — a bare pool bypassed the GUC.
- database/schema.go: migration tx declares a synthetic identity
  (schema.sql's dedup DO block SELECTs media_items; with policies
  live, boot 2 raised 'unrecognized configuration parameter' — A11).
- harness: TestServerSetup.DB is a raw-pool SEED handle (write
  policies permissive, other seeded tables ungoverned); the server
  under test keeps the scoped wiring. 33 CreateMediaItem sites use
  seedCtx() — a seed identity with no visibility rows — because
  RETURNING requires SELECT visibility of the new row.
- invariant test: regex UUID-scraping replaced with a structured JSON
  decode of the sections payload.

INTERRUPTION NOTE: an edit to this same effect (seedCtx) was applied,
confirmed by the tool, and subsequently vanished from disk while an
'opencode serve' process (PID 84551) is alive on the machine. This
commit pins the tree against further loss; implementation is PAUSED
pending the user killing that process.
2026-10-29 08:44:12 -04:00
John O'Keefe 004d4197a4 fix(test): rls fixture — reuse declared err variable 2026-10-29 08:26:10 -04:00
John O'Keefe f4b581872f wip(rls): ScopedDB + rls package + schema policies + echo bridge [pin compiling state]
Phase 3 of the RLS plan, pinned mid-flight because a parallel writer
was actively editing cmd/server/tests/rls_visibility_test.go (A14):
the file flip-flopped between two implementations and gained a compile
error after the previous commit. This snapshot makes the whole tree
compile; functional verification (two-boot gate, invariant green,
full suite) lands in the next commit(s).

- internal/rls: typed ctx keys — user identity (bridged from echo by
  the router) and an INTERNAL marker whose semantics grant nothing:
  no GUC is declared, so policy'd tables still raise at the engine;
  only RLS-ungoverned tables (system_settings, users counts) work for
  user-less startup paths (settings registry Load, CountAdmins).
- internal/database/scoped.go: DBTX decorator — per-statement tx,
  commit-on-Close (:many) / commit-on-Scan (:one), error paths always
  terminate the tx, refusal before SQL when no identity and not
  internal.
- database/schema/schema.sql: idempotent policy block (DROP IF EXISTS
  + CREATE — schema.sql re-runs every boot): ENABLE + FORCE, read
  policy exactly mirroring COALESCE(is_visible,true), permissive
  per-command write stand-ins (FOR ALL would OR into SELECT).
- router.go: RLS bridge at BOTH c.Set("user") sites (JWT + device
  auth) — echo-local storage never reaches the stdlib ctx.
- main.go + test harness: database.New over ScopedDB at both wiring
  sites; startup reads use rls.WithInternal.
2026-10-29 08:11:20 -04:00
John O'Keefe a19d7a4cbf test(visibility): fixture fixes from first live runs
Three harness corrections found while executing the invariant test:
- library type is 'ebooks' (plural) in library_types, not 'ebook'
- cleanup ordering: t.Cleanup is LIFO — register the pool Close BEFORE
  the fixture's library deletes, or the deletes run against a closed
  pool and fail silently into ignored errors (leaving RLSMARKER rows
  behind; found because a second run saw the first run's 'deleted'
  books still leaking through Recently Added)
- collection detail is addressed by the section's collection_id (the
  UUID in view_all_url), not the section name
2026-10-29 08:02:34 -04:00