65f257f377951016e73a4f2b8b4a940b9c857d00
368
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 John O'Keefe
|
65f257f377 |
fix(filter-builder): id labels resolve at render — refresh keeps names
Refresh showed raw uuids in id-kind value cells (feed/library/ collection/added_by). Two defects, one story: 1. Only ids persist in filter trees; the client's id→label cache dies with the page. The quiet re-resolution fetch reused the suggest facet, whose rows are MEDIA-CONSTRAINED (INNER JOIN media_items + the tree's sibling rules as a constraint) — so an id whose media doesn't match got no row and no label. Live repro: "feed in [A,B,C] AND title contains Chapter" labeled one feed of three (live-request evidence: 200 with total:1). A label is a property of the entity row, not of what currently matches. Fix: pages that render a tree resolve labels SERVER-SIDE at render and embed the map on the mount (data-id-labels) — the collection editor (stored tree) and the bookshelf (URL qf). New SearchService.FilterTreeLabels reads the entity tables with no media join and no tree constraint; scoping rides the request identity through the scoped handle (podcast_feeds/collections RLS — an invisible id resolves to nothing; pinned), libraries mirror the facet's visibility join. A feed with ZERO episodes resolves its title — pinned; that case was structurally impossible before. 2. The bookshelf drawer never hydrated at page load at all: the only load-time hydrate calls live in the PARENT's x-init (initBookshelf), and Alpine initializes parents before nested components — builderData() was null there, so hydrateFromForm no-oped and a URL-qf refresh left the drawer with an empty seeded rule while the grid stayed filtered. The drawer now hydrates in its OWN init when the mount carries no data-tree; the init-order contract is pinned in vitest (185/185). The quiet suggest fetch remains only as a fallback for trees the SSR never saw; embedded labels make it a no-op on refresh. Interactive menus are untouched (their constrained, count-bearing behavior is correct for choosing). Pins: TestRulesPageEmbedsIdLabels (zero-episode feed labels at render, bookshelf qf library label, hidden library NOT labeled); filter/episode-tree/search-contract/RLS/playlist container slices green; contract_pins entry documents the embed contract. |
||
|
John O'Keefe
|
81e54f89b2 |
fix(collections): the order select re-orders the PAGE — visible order = play order
The detail header's listening-order select persisted fine and the reader chain honored it, but the page's card list never applied chain_order (it kept the composed query's title order) — so flipping the select changed nothing visible, and Play as Sequence launches from the FIRST CARD ON THE PAGE, making the played start (and the perceived order) disagree with the curated setting (2026-11-01 user report on the smart podcast page). - Smart episode collections: the page's match list re-orders through OrderEpisodeMedia (the same OrderEpisodes comparator the sequence producer uses — §10 R2's single source of order truth) before card building. Books-scope matches keep title order. - Static episode playlists: the sequence now honors the curated order too — native keeps the manual position order (SQL), reverse flips it, oldest/newest re-key on publication date with feed position as tiebreak. Previously statics ignored chain_order entirely while the select rendered on their pages. Pins: native page order (feed positions) and the reverse flip re-ordering the SSR cards, asserted in the curated-order subtest alongside the existing reader-chain pins. Podcast/playlist/audiobook/ RLS/dashboard/contract slices green. |
||
|
John O'Keefe
|
cce488b76f |
fix(filter-builder): multi-pick id values matched nothing; hint the multi boxes
Comma-joined multi-picks matched ZERO: clauseFeed/clauseLibrary/ clauseCollection parsed each rule value as ONE uuid, but the builder's pick loop stores several picks as a single comma-joined value (["id1,id2"]) — uuid.Parse failed on the joined string, the id list came back empty, and "feeds in [A, B]" matched no episodes (one pick worked, which is why the briefing recipe survived). parseUUIDValues now splits every value on commas (trimming) before parsing, accepting both the builder's joined shape and hand-written separate elements — same leniency the suggest-side id lookup already has. The multi-value cells now say so: id-kind value inputs get the placeholder "pick one or more" (the only boxes on the page that accept several values; every other cell replaces its value). Pins: comma_joined_multi_feed_matches_both — a tree collection built with the exact comma-joined wire the client sends, asserting BOTH feeds' episodes match. Filter-builder vitest 35/35, container filter + episode-tree slices green. |
||
|
John O'Keefe
|
09f8f0875f |
fix(filter-builder): id labels survive Apply — stored ids resolve to names
The library cell reverted to the raw uuid after Apply: the apply flow reloads the page, the SSR-preloaded tree re-ingests, and idLabels — populated only by suggestion PICKS — starts empty, so valueCellText falls back to the id. Same on any fresh load of a stored id-rule. Two halves: - Server: the id-source suggest query's typed filter (library/ collection/added_by/feed) now also exact-matches a comma-separated ID list alongside the name ILIKE — so a suggest call with the stored ids returns their labeled rows. No new endpoint; the menu and the resolver share one surface. - Client: fetchSuggestions gains a QUIET mode (fills idLabels, touches no menu state), id rows fall back to their stored ids as the typed lookup when no draft is present, and init() calls resolveIdLabels() after tree ingest — one quiet fetch per id-kind rule resolves every stored id to its name from the first paint after reload. Pins: the Go side (suggest with a library's UUID returns the row with the library NAME as display), the client side (resolveIdLabels fills labels, the cell renders the name, no menu opens). 35 filter-builder vitest cases green; container filter slice green. |
||
|
John O'Keefe
|
7c7e282e79 |
feat(dashboard): custom-section builder saves again — the flagged tree migration
Archaeology, prompted by the user asking when it broke: the builder's save NEVER worked. The page was born (2026-02-19, |
||
|
John O'Keefe
|
a86bdac87e |
refactor(collections): retire the legacy rules surface — tree-only storage
The pre-tree rules machinery is deleted: no row ever carried a rules array or a non-default match mode (verified in the dev DB), the web never read the envelope keys, and the Android app consumes none of it. Dead code gets no documentation. Removed: - PUT /api/collections/:id/match-mode — endpoint, route, and the UpdateCollectionMatchMode query. The tree's root mode owns any|all. - auto_assign_rules / rule_match_mode from every envelope (create/list/detail/update), from the create/update request shapes, and from the schema: both columns DROPPED (schema.sql carries idempotent DROP COLUMN IF EXISTS heals for existing DBs; sqlc regenerated). - The rules CRUD service layer (AddRule/SetRuleEnabled/RemoveRule + validateRule + UpdateCollectionRulesJSON), ParseRules / EnabledRules / IsSmartCollection, RulesToFilterTree (+ the SSR preload branch), the legacy branch of EvaluateSmartMembership, and SmartEvalOptions.MatchMode. IsCollectionSmart = tree presence; create-path isSmart = collection_type=smart OR a filter_tree. Kept (live consumers): POST /api/collections/preview still evaluates rules payloads in memory (the custom-section builder's preview), and include_auto stays on the list (the Android app sends it). The custom-section builder's SAVE was already broken — its stringified auto_assign_rules payload failed request binding (probed live: 400 invalid request). Create now rejects that payload loudly with a 400 naming the retirement, instead of silently creating a hollow static section; custom-section-builder.md carries the flag. The builder's tree migration remains the flagged follow-up. Test-side (asymmetry rule — deleted behavior, deleted tests): TestPodcastSmartCollections rewritten onto trees (mode lives in the tree root), match-mode subtests and tree_wins_over_legacy_rules deleted, TestRulesToFilterTree deleted, contract-pin envelope keys retracted per the contract_pins 2026-10-31 retirement entry. Mirrors hand-synced: openapi.yaml, bruno create/update, docs/api/*.md, docs/schema.go, api.d.ts. Full container suite green (155s). |
||
|
John O'Keefe
|
ab98015190 |
feat(collections): RLS ownership enforcement at the DB layer
collections + collection_items join the RLS-governed tables, same doctrine as the media_items family lens (2026-10-29). Policies route through two PURE predicate functions (app_can_read_collection / app_can_write_collection) that never query collections — no policy recursion under FORCE RLS — so a future sharing model is one EXISTS branch inside the functions, not a schema re-touch. Reads: owner or admin; writes: owner, admin escape for SYSTEM collections only. Why: the write path had no app-level ownership gate on Update/Delete/ AddBooks — any authenticated user could mutate another user's collection by id. The DB is now the enforcement of record; existing app-level gates (match-mode, reorder, view-settings) stay as defense-in-depth. Behavior changes (contract_pins 2026-10-31): - cross-user GET detail: 200 -> 404 (row invisible) - cross-user PUT update: silent success -> 404 (ErrNoRows mapping; service wrap fixed %v -> %w so errors.Is sees pgx.ErrNoRows) - cross-user POST books: silent success -> 404 - cross-user DELETE: 204, now a true DB no-op - anonymous sees nothing; admin persona unchanged Bootstrap: registration seeds system collections BEFORE the JWT exists — CreateDefaultCollectionsForUser declares the new user's identity (rls.WithUser), the same fix mirrored in the test harness (createDefaultCollectionsForUser owner-scoped; collection sweeps run under the owner's identity). Tests: new TestCollectionsRLS pins the cross-user matrix end-to-end (404 detail/update/add/view-settings, 204 no-op delete, admin escape, owner positive control, registration seeding under RLS). Harness fixture writes to non-system collections now run as the row's OWNER (scopeExecAs/ownerSeedCtx) — the admin write-escape covers system rows only, and bare-ctx pool reads of collections see nothing. Filter/playlist/audiobook/scan-state fixtures converted. Full container suite green (159s). |
||
|
John O'Keefe
|
536fb4794c |
fix(collections): the create modal's seed POST 400'd — echo cannot bind a form string into an any-typed field
The first real-browser create of an Episodes-scope collection failed with "invalid request": the modal's hx-post submits application/x-www-form-urlencoded, and the form value for filter_tree had to bind into CreateCollectionRequest.FilterTree (any) — echo's form binder errors on any-typed fields instead of setting the string, so every form carrying a tree 400'd before validation even ran. JSON bodies (the save-filter bridge, the editor's Apply) were unaffected, which is why no API-level or Go test caught it: only a browser posting the real form shape exercised the transport. Fix: the form variant binds a dedicated string field (FilterTreeForm, form:"filter_tree") on both request structs while FilterTree keeps the JSON tag; resolveTreePayload picks whichever transport arrived and both ride the same parseRequestFilterTree validation. Pinned by TestEpisodeCollectionsFormSeedCreate (form-encoded seed → 201 → scope round-trips). |
||
|
John O'Keefe
|
cc0e9d7f91 |
feat(podcasts): the episodes authoring door + curated chain order
"Newest episodes from my selected podcasts" is now buildable in the UI, end to end — the user could not find ANY podcast-related filter surface because none existed: the New Collection modal could only create book-scope smart collections, and the engine's episodes scope (item 3.1) had no authoring door. The scope door: - The create modal's Smart card gains a Rules-scope segmented control (📚 Books / 🎧 Podcast episodes); the copy no longer claims smart means "match books by rules". Choosing Episodes seeds an EMPTY episodes tree ({"scope":"episodes","mode":"all","children":[]}) so the collection is born episode-scoped: the editor opens with the episode vocabulary (feed/downloaded/played/published date) and membership runs the episode engine from the first rule. - ParseFilterTree now accepts an EMPTY ROOT (nested empty groups stay invalid) — the seed's legal shape. The root arrives at depth 1 by call convention, hence the depth>1 exemption. - The composers define the empty-tree semantics: zero children matches NOTHING (ComposedLiveMedia/ComposedCount guards), mirroring EvaluateSmartCollection's zero-enabled-rules convention. Without the guard the composed query degrades to the bare scope head and a fresh collection would swallow every episode in scope — pinned by TestEpisodeCollectionsEmptySeedMatchesNothing (envelope book_count 0) and TestEmptyRootTreeSeedsEpisodesScope. The curated chain order (plan §10 R2, user need: the briefing plays NEWEST-first): - view_settings.chain_order ("native"/"newest"/"oldest"/"reverse") stored on the collection — curated content, not a player-side knob. - PUT /api/collections/:id/view-settings merges the key into the stored JSONB (a blind write would clobber the playlist toggles). - buildSmartCollectionSequenceContext routes OrderEpisodes through handlers.CollectionChainOrder; the detail header's Play-as-Sequence gains the order select next to it (PlayCollectionButton), which PUTs and reloads. Pinned by chain_follows_the_curated_order (reverse flips the chain indexes and hops). Caught by the new test, a latent clobber fixed: UpdateCollection treated a NIL view_settings payload as "write NULL" — any edit that carried no settings dropped every stored key (chain_order, playlist toggles from API clients, imported provenance). Nil now means untouched, the same convention filterTree already had. Composer guards also cover ComposedCount, so the builder's live count reads the same semantics. Tests: collectionChainOrder unit matrix, empty-root parse unit pin, two integration pins (seed matches nothing; curated order drives the chain and survives whole-map edits). |
||
|
John O'Keefe
|
4f71eda6be |
feat(podcasts): the rules page retires — the collection builder IS the filter builder (item 3.4)
The smart-collection editor at /collections/:id/rules now hosts the SAME builder island as All-Books (FilterBuilderRows — the rows markup extracted into one shared templ both mounts render; the shared filterBuilderExtra component). One vocabulary, one engine, one row UI; the collection differences are exactly three: the payload is the collection's scope (episode trees get the episode fields — books trees hide them), Apply writes the tree to the collection (PUT filter_tree through the validated write path), and a legacy rules array preloads as its CONVERTED tree (RulesToFilterTree rendered server-side) so the upgrade lands at the first Apply — read-compat never mutates data silently. Retired with the page: the per-rule CRUD endpoints and test-rules (their handler bodies, routes, api-docs rows, and the dead collections.ts island functions — the file's LIVE halves, the collection modal/navigate/ grid-refresh, stay; the dead rules half went). match-mode endpoint and preview survive (legacy channel + the dashboard custom-section-builder, flagged for its own migration). Contract-pin changelog documents the deletions per the asymmetry rule. The builder component gains collection mounts: scopeKey (the row universe for counts/apply), data-tree prefill, treeJSON (the apply wire with scope riding the top level), and an apply hook (the page owns the write; All-Books never sets one). ensureReadyRule defaults to the scope's first field instead of hardcoding author. Tests migrated onto trees: TestSmartCollections' live-membership / reject-mutations / include-auto / system-collection subtests now drive TREE collections; the deleted-endpoint subtests die with them. |
||
|
John O'Keefe
|
6b2125a45a |
feat(podcasts): smart collections ride the unified filter engine — filter_tree column (item 3.2)
collections gains filter_tree JSONB (idempotent in-place; same qf DSL as saved filters — the scope key included). IsCollectionSmart is the ONE detection (tree OR legacy rules), and EvaluateSmartMembership is the ONE evaluator: tree-bearing collections run the composed engine via ComposedLiveMedia (full media rows, so the collection surfaces keep their episode-card stamping, feed positions for the replay chain, and the family lens rides the mandatory head), while legacy rule arrays keep per-item FieldMatches until rewritten — one call surface, two storage epochs. All smart surfaces (detail JSON, counts, detail page, reader chain) dispatch through it; the search service is injected into the collection subsystem (variadic — main.go builds one shared instance; the test harness mirrors it). Write path: CreateCollection/UpdateCollection accept the tree, validated against the engine grammar at the HANDLER (an invalid tree is a 400, never a stored render-time 500) and exposed in the envelope (pin widened in TestContractPin_CollectionsJSON + changelog). Update semantics: an edit form that doesn't send a tree keeps the stored one. RulesToFilterTree converts legacy rules + match mode at upgrade time: episode rules map mechanically (feed→in-list, played/downloaded→ booleans, published_within→published_date+within_days), tree root mode absorbs rule_match_mode, episode rules force the episodes scope, and legacy starts_with/ends_with fall back to contains (documented). Read-compat by design: nothing is migrated at boot — legacy arrays evaluate exactly as before until the collection is rewritten. Pinned by TestRulesToFilterTree (unit conversion) + TestEpisodeCollectionsFilterTree (a tree collection evaluates through the engine, the chain follows the tree's live membership, invalid trees 400 at write, the tree wins when legacy rules coexist, and an edit without a tree keeps it). |
||
|
John O'Keefe
|
56871b65ed |
feat(filters): scopes + episode fields — the engine becomes the single query system (item 3.1)
The qf tree gains an optional top-level `scope` ("books" default |
"episodes" — absent = books, so every existing saved tree and legacy
client stays valid). The row head is scope-aware AND this closes a
real leak: the composed engine's head had NO podcasts exclusion, so
episodes already surfaced on All-Books qf queries (plan §9 violation —
the sqlc flat head excluded them, the composed head didn't). Books
scope now says library_type_name <> 'podcasts'; episodes scope is
source = 'podcast_episode'.
The registry fields declare their scopes (the plan's "reusable
template with conditionals" = scope-driven rendering in 3.3): episode
fields declare exactly the episodes scope and the books scope REJECTS
them at parse (the "hide podcasts from the rules on the all books
page" rule enforced server-side); books fields inside episodes trees
are allowed and honestly no-op on NULLs.
New fields (the item-2.4 rule vocabulary expressed in SQL):
- feed — KindIdentifier over podcast_feeds (RLS-governed suggestion
source and match; unsubscribed feeds match nothing, by the engine)
- downloaded — is_true/is_false on download_state = 'downloaded'
- played — THE canonical finished predicate (viewer progress >= 0.999,
the hide_finished definition shared with item 2.4), reading_progress
join per viewer
- published_date — clauseDateFactory like added_at, within_days
default op (the published_within rule's SQL form)
Engine hygiene: the stray ComposedSearch SQL debug printf is gone.
Pinned by TestFilterScopeValidation (unit: scope grammar, books×
episode-field 400s, legacy no-scope trees) and
TestFilterScopesAndEpisodeFields (integration: books head excludes
podcasts including the 404-no-results contract, episodes head is
episodes only, feed-in/downloaded/played/published-date evaluate in
SQL against a subscribed-feed fixture with a played viewer).
|
||
|
John O'Keefe
|
bc91ce68d4 |
test(pins): widen whitelist for the episode-collections JSON shapes (item 2 series)
TestContractPin_CollectionsJSON tripped on the additive keys the episode-collections work introduced — the tripwire worked exactly as designed (additive keys must be whitelisted here AND in docs/contract_pins.md first). - Collections envelope gains rule_match_mode (item 2.4): the smart-collection match semantics on GET /api/collections/:id. - BookInfo gains four STABLE keys (item 2.3): is_episode, feed_title, download_state, hidden — dropped omitempty so every card surface emits the same key set (the viewer_percentage precedent; adapters that predate episode awareness emit zero values). docs/contract_pins.md changelog entry written. Nothing removed, nothing renamed. |
||
|
John O'Keefe
|
69773ec307 |
feat(podcasts): smart collections play as a sequence — the Daily Fiction acceptance (item 2.5)
ACCEPTANCE (plan §0 item 2.5): "Daily Fiction" = feed ∈ {A} ∧ unplayed
∧ <7d matches exactly today's episodes, and PLAY = today's episodes
back-to-back. Two pieces were missing after item 2.4:
- The chain builder only read static membership (collection_items) — a
smart collection has none. buildSmartCollectionSequenceContext now
evaluates the rules LIVE for the requesting user (mode from the
collection, family lens via the visible set) and chains the
downloaded episode matches in NATIVE feed order — the collection's
computed order is content (R1: never written back; OrderEpisodes is
the order chokepoint per R3). A match that aged out of the briefing
(played since, published beyond the window) falls back to the feed
queue instead of breaking the page.
- The smart-detail branch rendered match cards as books: episode
matches are now flagged (IsEpisode + download state) so they carry
the play affordance and the header's Play-as-Sequence button;
feed titles stay empty on smart cards for now (batch lookup is
polish, not acceptance).
Pinned by TestPodcastSmartCollections' acceptance subtests: the
briefing starts at index 1/2 and chains into the second with the
queue param on both hops; the order control stays hidden (computed
order); a stale match falls back to the feed queue honestly.
|
||
|
John O'Keefe
|
abb7c84833 |
feat(podcasts): smart episode collections — ALL/ANY match modes + episode rule fields (item 2.4)
The daily briefing needs AND: today's rules engine was OR-only. The
match semantics become a per-collection column
(collections.rule_match_mode, 'any' legacy default | 'all'), flipped on
its own endpoint (PUT /api/collections/:id/match-mode, owner-only —
nothing else on the row changes) and surfaced in the rules page as an
ANY/ALL toggle with matching copy.
Episode rule fields join the vocabulary (plan item 2.4): feed (the
multi-select rides the new `in` operator — comma-separated feed ids),
played (the canonical finished predicate — viewer progress ≥ 0.999,
same as hide_finished), downloaded, published_within (N days), plus
title/description contains. The rules candidate query now carries the
viewer's progress (LEFT JOIN reading_progress) so played/unplayed is
per-user, and TestRules/preview speak the SAME evaluator: the handler's
duplicated field switch is deleted and delegates to
CollectionService.FieldMatches — one vocabulary, one semantics.
Legitimate semantics note: in ANY mode `played:false` matches unplayed
BOOKS too (everything with no progress row) — correct per-rule
behavior, and exactly why the briefing mode exists.
Backward compatibility: every legacy row defaults to 'any' (the old
OR), UpdateCollection keeps the stored mode when the form doesn't send
one, and the evaluator treats absent data honestly (no date → no
match; no progress → unplayed).
Pinned by TestPodcastSmartCollections: the AND-mode briefing
(feed∈{A} ∧ unplayed ∧ <7d = exactly the new episode), OR-mode
widening, owner-only flips, the preview honoring the mode, and
downloaded/contains rules. Also fixes a latent bug the new query
surfaced: UpdateCollectionViewSettings referenced a nonexistent
updated_at column (never exercised until now — the new sibling query
was written without it).
|
||
|
John O'Keefe
|
adb93cba9e |
feat(podcasts): episode collections — picker scope, hint rows, the chain (item 2.3)
D11 lifts: podcast episodes (media_items rows) join collections, and "Play collection as sequence" feeds the player from the collection's order. No new tables — collection_items already carries ordered membership (item 2.1 audit). PICKER: the collection modal gains a Books/Episodes tab pair. The episodes tab hits /api/media-items/search?scope=episodes — a dedicated picker query (SearchEpisodesForPicker) whose visibility is entirely RLS: the media_items subscription predicate drops unsubscribed feeds' episodes, and the podcast_feeds join is governed the same way. The books scope keeps its podcasts exclusion (plan §9) — this scope is the only door in. Pending episodes are pickable (plan ahead; the detail page shows them as not downloaded). Selection rides the existing bookPicker store — ids never collide across scopes, and AddBooks works unchanged on episode ids. DETAIL: the All-Libraries membership view switches to GetCollectionItemsDetailed — LEFT JOIN + per-row visibility. An unsubscribed episode (or family-lens hidden book) survives as a Hidden card rendered as an "Unavailable — remove entry" hint row instead of silently vanishing; visible episodes render feed title, download state, and a Play link. The library-scoped view keeps its existing dashboard-query behavior (the switcher's contract is untouched for books). CHAIN: ?queue=collection:<id> on any reader URL makes buildPodcastSequenceContext produce the COLLECTION context (§10 R2 same shape, source=collection): downloaded episodes in collection order (positions are content — R1; no feed-order override), prev/next URLs carrying the queue param so the chain survives hops, and backToDetail landing on the collection page. Ownership gates the chain — foreign/deleted ids fall back to the feed queue; the order control stays hidden (2.2's source gate). The header's "Play as Sequence" button starts from the first downloaded episode in list order. Pinned by TestPodcastEpisodeCollections (7 subtests): RLS-scoped picker + family-lens auditor case (subscribed episode in a hidden library must not reach the picker), books-scope exclusion intact, hint rows, chain carrying the queue param, start-from-first, foreign fallback, and reorder driving the chain (the R4 primitive works on episode ids unchanged). |
||
|
John O'Keefe
|
a90e7debfd |
feat(podcasts): queue context source + player order control — item 2.2
Foundation rule R2 (§10): the queue context is the only queue, and every producer emits the same shape. PodcastSequenceContext gains source (feed|collection), feed_id, and order; the feed producer sets them now — the collection producer (2.3) slots into the same shape without the player ever special-casing beyond showing/hiding controls. The player header gains the FEED-mode order control next to the "Ep N/M" counter: a dropdown (native/reverse/oldest/newest) plus a one-click reverse button. Both PUT the existing per-user feed order override — the SAME state the feed page writes, so flipping it in the player updates the feed page's dropdown too. The control renders only when source=feed: a collection queue's order is curated content (R1), never a listening preference. Behavior is hx-put + hx-swap="none" (response discarded) with a module-level htmx:afterRequest listener in audio.ts reloading the page — the server re-renders prev/next/index in the new order; templ decides what it looks like, JS only triggers the round trip (CSP-clean, no inline handlers). Pinned by TestPodcastPlayerOrderControl: the config JSON carries source/feed_id/order/episode_index; the control renders feed-mode; an order override re-renders with the dropdown reflecting it (data- json pins must match templ's " attribute escaping). |
||
|
John O'Keefe
|
7bf0b1257d |
feat(podcasts): widen reader_settings key ceiling — item 2.1 foundation
reader_settings.setting_key was VARCHAR(50), and the composite podcast_order:<uuid> key already sits at exactly 50 chars — zero headroom. The deferred queue-editor idea (plan §0 item 2b) and any future per-feed/per-collection preference key would have been forced to payload-stuff inside the order row by a column width. Composite prefix:<uuid> keys are the norm now (§10 R5), so the ceiling widens to VARCHAR(120): idempotent in-place ALTER (varchar widening is metadata-only, no table rewrite), re-runs harmlessly on every boot per the schema convention. No new episode-membership tables: collection_items already carries ordered playlist membership (position, A2c) against media_items, which podcast episodes ARE — episodes join collections through the existing table. Pinned by TestReaderSettingsCompositeKeyHeadroom: a 57-char composite key (the podcast_custom_order:<uuid> shape) upserts and reads back. Under the old width this fails; sqlc codegen is byte-identical. |
||
|
John O'Keefe
|
f47734dd7d |
fix(podcasts): generic media delete refuses episode rows (invariant 2)
The admin endpoint DELETE /api/media-items/:id predated podcasts and would delete any media row — including podcast episodes — if hit directly. Invariant 2 (absolute form, user decision 2026-10-30) says episode rows are removed ONLY by whole-feed removal: "the feed should match the XML until the entire feed is gone." The plan audit found this endpoint was the one remaining surface that could violate it: the poller, unsubscribe, and the admin file-deletion endpoints are files-only, and the hash-conflict merge can't see episodes (they never hash). The guard sits right after the GetMediaItem read and BEFORE the storage-down check, so a refused episode never touches storage resolution. Same segregation pattern as the generic edit path (D9/D10, UpdateMediaItem). Pinned by TestDeleteMediaItemPodcastGuard: admin DELETE of an episode → 409 with the row surviving; a book delete through the same endpoint → 204 control. |
||
|
John O'Keefe
|
7d5bbd1177 |
feat(admin): multi-select episode file deletion — checkboxes on downloaded rows
The plan's per-EPISODE bulk select for the admin view (§0 item 8 follow-up — trimmed in the first admin-view commit; user called it back). Admins now tick checkboxes on DOWNLOADED episode rows and delete their files in one request: - Checkbox per row, rendered ONLY for admins + downloaded episodes (nothing to delete otherwise), via the existing podcastDetail Alpine component: selection ids/count survive the list's htmx swaps (reapplied on afterSwap — same pattern as the text filter) and clear after a successful bulk delete. - Toolbar segment outside #episode-list (survives swaps): 'N selected' + 'Delete selected files' (hx-include gathers the CHECKED boxes); disabled-looking at zero selection; hx-confirm states the re-materialization contract. CSP-safe method bindings only (bulkSelectionLabel/Class) — no ternaries in Alpine expressions. - POST /feeds/:id/episodes/delete-files (admin): repeated episode-file-select keys parsed via ParseForm directly (never bound — repeated-key wire is another echo-bind gotcha); each id validated to belong to THIS feed, so a forged foreign id is refused while the real selections apply; rows reset to pending, files removed. - Tests: bulk of two real ids + one forged id → deleted:2, files gone, rows pending; non-admin 403; all prior single/feed-level pins hold. |
||
|
John O'Keefe
|
0741e98ea5 |
feat(admin): all-podcasts view — disk-first maintenance for every feed
§0 item 8 (Phase-4 item 1 pulled forward, user directive: maintenance
tooling ships WITH the features it maintains):
- /admin/podcasts (AdminMiddleware): every feed from EVERY user via the
RLS admin escape (first-admin identity, poller/fetcher contract) —
feed + host, library, subscriber count/names, downloaded/total, DISK
USAGE, last fetch. Sorted disk DESC ('what's eating the disk'),
totals header (feeds · episodes · bytes on disk).
- Per-feed actions: Refresh · Delete files (rows → pending, disk freed;
confirms spell out the re-materialization contract) · Remove feed
(rows + files for everyone — the same admin delete as the detail
page). All confirms ride the global confirm modal.
- New sqlc: ListAllPodcastFeedsAdmin (correlated aggregates),
ResetPodcastEpisodeFileDeleted / ResetPodcastFeedFilesDeleted —
downloaded rows reset to pending with download-derived facts cleared
(audio_metadata, chapter_count, cover, size, mime); file_path KEPT
(fetch-maintained layout — a re-download lands on the same path).
file deletion via the existing DeleteFeedFiles file-removal pattern
(episode files + .cover.jpg siblings).
- Poll upsert never touches download_state, so deleted files STAY
deleted — a feed refresh re-lists metadata but re-materializes
nothing (pinned by TestAdminDeleteEpisodeFiles).
- Per-episode admin delete-file: POST /feeds/:id/episodes/:eid/
delete-file (admin) — one episode's file gone, row pending, siblings
untouched; HX swaps the episode list. (Found while testing: the
handler first read c.Param("id") — the FEED's uuid — instead of
:eid; the 404 'episode not found' trail led straight to it.)
- Tests: admin sees other users' feeds + subscriber names; regular
users 403 on page AND actions; single + feed-level deletion reset
rows/clear facts/delete files; refresh leaves pending rows pending.
- Admin nav gains 'Podcasts' under Administration.
Wrong-gotcha note for future agents: parseUUID on c.Param('id') inside
a /feeds/:id/episodes/:eid route resolves the FEED, not the episode.
|
||
|
John O'Keefe
|
15f41e6fd4 |
test(pins): whitelist download_queue_pos on raw media rows (additive widening)
The contract pin caught the §0 item 4 column surfacing on media-list raw sqlc rows — its job. Widened additively per doctrine: rawRowKeys + docs/contract_pins.md changelog entry. NULL for scan rows and never-queued episodes; nothing removed or renamed. |
||
|
John O'Keefe
|
5c545534c2 |
feat(podcasts): Download All queues in the LISTENING order + add-modal order dropdown
Download All previously queued in feed-native order — 'getting the last chapter first doesn't help me'. Now the queue follows the feed's effective listening order (§0 item 5b): QueueFeedDownloadsAs reads the CALLER's order override and runs episodes through services.OrderEpisodes (same helper as the sequence/reader/page surfaces — the consolidation from item 5a landed with the reverse-order commit). Queue tickets (SEQUENCE-stamped, item 4) make that enqueue order the claim order, so chapters land in the order you'll actually listen. Add modal (§0 item 5c): with 'Download all episodes now' checked, a 'Download order' select appears (pure-CSS :checked sibling reveal — no JS island, CSP-clean). Its choice is the feed's INITIAL order override (reader_settings upsert; 'feed' = no row) AND the queue order for the initial batch. Per-book by design: the override IS the download order. Test TestPodcastAddModalDownloadOrder: add with download_all+reverse → the override row exists AND persisted tickets run position 3→1. Tickets are asserted (not live queued rows) because the shared dev server's worker legitimately races this test's queue — tickets survive every state flip. |
||
|
John O'Keefe
|
0f3c4a8cec |
feat(podcasts): concurrent download pool — fair share, tickets, admin tunables
Replaces the sequential worker with a download-manager-style pool (§0 item 4). Model: the user's connection is the limit; semantics are fair-share ACROSS feeds, not goroutine-per-book (a global FIFO would make Book B wait behind Book A's 100 files). - Atomic claim: GetNextQueuedEpisode + SetPodcastEpisodeDownloading was a read-then-write race under concurrency; ClaimQueuedPodcastEpisode converts queued→downloading in ONE guarded statement (two claimants, one winner — the loser gets ErrNoRows). SetPodcastEpisodeDownloading retired. - Queue tickets: download_queue_pos BIGINT fed by a Postgres SEQUENCE stamped at enqueue — the enqueue loop's ordering IS the claim order (the foundation §0 item 5 builds on; previously all rows in a batch shared one updated_at, so within-feed order was arbitrary). - Pool: up to podcast_download_concurrency (default 4, live range 1–16) claim-loop goroutines per drain round; the setting re-reads every round so admin changes take effect without restart. - Fair share: claim prefers the feed least-recently-served (pickFeedToServe; never-served first), earliest ticket within a feed — books proceed in parallel while chapters land in order (start listening while the rest download). - Bandwidth: shared token bucket across all workers (podcast_download_speed_mbps, default 0 = unlimited); take() reserves bandwidth and callers sleep outside the lock. - Settings: new float setting type end to end (registry defaults + clamping, save validation, admin UI number-with-step input) + the two schema seeds; worker wires the registry via SetSettings (tests run on registered defaults). - Tests: unit coverage for the bucket reservation math + fair-share pick; integration TestPodcastDownloadPool pins ticket order = enqueue order, claim exclusivity, and a full 3-episode pool drain. Dev-DB note: the column/sequence/settings were applied live (idempotent DDL, same statements the schema runs at next boot) since the running container predates them. |
||
|
John O'Keefe
|
e3675536aa |
feat(podcasts): reverse order — position DESC, date-agnostic; single order helper
The 'Oldest first' order sorts by PUB DATE, which breaks on Scribl's migrated podiobooks feeds (messy/all-equal dates). New order value 'reverse' = feed position DESC with a date-DESC tiebreak — dates are not involved, so nothing can break. - services.OrderEpisodes: THE single order source of truth (§0 items 3+5a). The four surfaces (sequence JSON, reader chain context, SSR page list) previously carried three near-identical handwritten comparators — all now route through one generic helper. The SSR page list gains a SortDate key (lexical 2006-01-02) so the pub-date tiebreak is actually chronological; the old comparator compared 'Jan 2, 2006' display strings alphabetically (latent drift bug). - Dropdown becomes four options: Feed order / Feed order (reversed) / Oldest first (pub date) / Newest first (pub date). - Serial suggestion button renamed 'Reverse feed order' (sets reverse, not oldest) and its response out-of-band-swaps the order select (PodcastOrderSelectOOB — same mechanism as Download All's policy select), so the dropdown mirrors the server state immediately. - Validation: oneof=native reverse oldest newest everywhere. - Tests: reverse binds via the form wire; messy-dates sequence returns 3,2,1; existing reorder-safety stays green. |
||
|
John O'Keefe
|
7c7c08748e |
fix(podcasts): download_all binds on every wire — FlexBool + flip log
Reproduced the §0 item 2 suspect: echo v5 bool-binds form values via strconv.ParseBool, which accepts "true"/"1" but REJECTS the browser-native <input type=checkbox> value "on" — and a bind failure kills the WHOLE AddFeed request (400 "invalid request"), not just the checkbox. The add modal posts urlencoded (htmx), so any checkbox-serialization that emits "on" made Download All silently unusable. Fix: FlexBool (encoding.TextUnmarshaler-style UnmarshalParam) binds download_all permissively on every wire — "true"/"on"/"yes"/"checked" => true, explicit false/0/off => false, ABSENCE (unchecked box on any wire) => false; JSON bool binding is native and unaffected. Tolerates the exact wires htmx and browsers emit. AddFeed's subscription flip now logs "[PODCAST] flip ... <feed> — user" so the live flip is provable in app logs (user suspected it never fired). Integration test TestPodcastDownloadAllFormEncoding pins the urlencoded wire (what htmx actually sends): true + on both flip the caller's policy to 'all', an unchecked add leaves the default. |
||
|
John O'Keefe
|
7ea84043fa |
fix(auth): logout actually signs out — clear the HttpOnly session cookie
Release / build-and-push (push) Successful in 3m9s
Logout wiped localStorage but never cleared the token cookie (HttpOnly — only a server response can) and never called the logout API, leaving the refresh token live for 30 days and every cookie-authenticated SSR page accessible: the fake-signout bug. Reproduced on prod and dev; unrelated to RLS (it sits behind authentication) and long-standing. - /api/auth/logout now ALWAYS clears the session cookie (Set-Cookie empty + Max-Age=-1, HttpOnly/Path/SameSite mirroring login) — even a malformed body must sign the browser out; refresh-token revocation unchanged. - The header logout button (and the rules-page copy) now POST to the API with the stored refresh token BEFORE clearing local state and redirecting — best effort: local cleanup never blocks on API failure. - toast 401 handler fixed: it cleared 'refreshToken' but storage uses 'refresh_token' — the real key. - Regression test TestLogoutClearsSession: real login → logout → response deletes the cookie; logged-out refresh token can't mint access tokens; body-less logout still clears. Also carries regenerated artifacts from the confirm-modal and episode-list work (header_templ/podcast_detail_templ/style.css). |
||
|
John O'Keefe
|
253d8d0e8e |
feat(podcasts+ops): Apple launcher-scheme feeds + visible build stamp
Release / build-and-push (push) Successful in 3m29s
Apple scheme normalization (follow-up to the Scribl discussion): Add Feed accepts the launcher spellings podcast pages ship next to plain RSS — itpc://, podcasts://, feed:// — and rewrites them to the https feed URL they wrap (scheme match case-insensitive). http stays and the guard still rejects it. Unit-pinned. Build stamp (stale-container drift guard): the 2026-10-29 podcast session lost an hour to a container running a build that predated the feature. version.Commit rides ldflags from a GIT_COMMIT build arg; the dev compose passes git rev-parse --short HEAD via the Makefile; the app logs 'Bookhoard <ver> (commit <hash>)' at boot and /api/health reports it — the UI and the repo can no longer silently diverge. |
||
|
John O'Keefe
|
30766020c8 |
feat(podcasts): real-time episode list — byte progress bars, direct swaps (issues #1/#3)
WHY: downloads needed manual refreshes to see progress; the 204 + HX-Trigger event indirection missed its listener, and the worker buffered whole enclosures so no progress existed to show. - Worker streams enclosures to disk through a counting loop (64KiB chunks, scheme/redirect/size guards preserved) with an in-memory per-episode progress snapshot (received/total; total from Content-Length, seeded by the feed's declared length). Deliberately transient — the DB row state remains the durable truth. - Episode-list renderer seam: the router injects a closure into PodcastHandler (handlers can't import templates — MediaDetail cycle) so htmx download/retry/Download All/refresh/order/policy actions answer with the REFRESHED partial directly (200, hx-target= #episode-list) — no events to miss. Non-HX callers keep JSON; tests without a renderer keep 204+trigger. - Polling tightens to 2s while downloads are active; downloading rows render a percentage bar + MB counter (quote-free inline style per templ gotcha #8) or an indeterminate bar when the server sends no length. - Issue #3: Download All (button AND the add-feed checkbox) flips the CALLER's subscription policy to 'all' — per-user, never global (D6: files shared, policies not) — and the auto-download select out-of-band-swaps so the dropdown shows what the user is getting. They can turn it down afterwards; the flip is honest communication, not coercion (user-approved 2026-10-29). PINs: Download All with HX answers 200 + episode-list + oob select and flips newest → all; add-feed download_all sets policy all. |
||
|
John O'Keefe
|
ffc80f7ba7 |
fix(podcasts): page search hits episode titles/descriptions (issue #8)
WHY: the /podcasts search box filtered feed CARDS only (title, display name, host, publisher, feed description). D14's episode search existed only as the /api/podcasts/search endpoint and was never wired into the page — searching 'Quarter' (present only in episode titles) cleared the grid while 'Trader' (feed metadata) survived. The page query now consults SearchMyPodcastEpisodes (subscription-scoped via RLS — the caller's identity) when a query is present; a feed surfaces when its OWN metadata OR any of its episodes matches, with an 'N matching episode(s)' badge on the card. PIN: SSR test with a needle that exists only in an episode title — the feed must surface with the match count. |
||
|
John O'Keefe
|
c753780817 |
fix(podcasts): order selector validation — bind the htmx form wire format (issue #2)
WHY: htmx sends non-GET requests urlencoded by default; the SetFeedOrder request struct only carried a json tag, so the binder left Order empty and 'required' fired — the exact error the order dropdown AND the Serial-suggestion button produced (hx-vals JSON also arrives form-encoded). One form tag fixes both controls. Regression test drives the exact wire format: urlencoded PUT with order=oldest/native → 200, empty/invalid → 400. |
||
|
John O'Keefe
|
5c10bbb90e |
test(podcasts): reorder-safety pin — progress survives position renumbering (task 2.6)
Invariant 3: progress anchors on the episode row (guid identity + chapter/offset), never the ordinal. Renumbering feed_position (the fetcher's per-fetch renumber case) leaves the progress row untouched and the sequence membership stable — downloaded-only with no gaps. |
||
|
John O'Keefe
|
49b0b6eeb9 |
test(podcasts): integration suite + RLS auditor + contract widening (task 1.10)
cmd/server/tests/podcast_test.go — six suites, the shared-dev-DB doctrine (test*-named libraries, cascading cleanup, no binary fixtures): - TestPodcastRLSSurfaces: THE AUDITOR (invariant 13) — non-subscriber 404 on feed detail + SSR page (existence leak), episodes absent from generic media reads, admin escape, subscriber visibility, then the hidden-library flip hiding an already-subscribed feed (the feed policy's library-visibility join). Seeding runs as the FIRST ADMIN — the seed identity has neither subscription nor users row, so podcast INSERT..RETURNING (SELECT policy under FORCE) 42501s for it; production pods run as exactly that admin (invariant 12). All direct row probes read through the admin scope for the same reason. - TestPodcastAddFeedFlow: member add + subscribe + metadata-only upsert; duplicate-URL dedupe (no second row); allow_member_feeds gate (member 403 / admin 201 / subscribe-still-allowed). - TestPodcastShrunkenFeedRetention: feed purges an episode from the XML, both rows persist (upsert-without-delete pin, D14). - TestPodcastEditSegregation: generic edit 400 + write-to-file refusal for episodes; control book edit unaffected. - TestPodcastDownloadWorker (ffmpeg-generated tone at runtime): file lands at <download-root>/<feed-slug>/NNNN-…, audio_metadata probed; re-download of a downloaded episode never re-fetches (idempotency pin, enclosure-hit counted); dead enclosure → failed + persisted reason + metadata intact. Fetch/SSRF tests swap the client via SetPodcastHTTPClientForTests (the production client refuses loopback httptest servers by design — the real client gets its own negative test in services). - TestPodcastScannerExclusion: SetFolders on a podcasts library root yields zero scanner folders (invariant 4 pin) — MediaScanner .FolderCount() accessor added. Unit: internal/services/podcast_rss_test.go — namespace-aware parsing fixture (three XML namespaces, fallback chains, hosts/guests), duration spellings, RFC-822 variants, explicit variants, slug/path layout, enclosure-extension selection, stripHTML, and the SSRF matrix (private/loopback/link-local/multicast/mapped-v6 blocked; a loopback httptest server PROVES the production client refuses it). suites green: full integration run 161s / 0 failures; vitest 12 files 163 tests. Contract widening: TestContractPin_MediaListBothBranches' raw-row allowlist gains the 15 podcast episode columns (additive; documented in docs/contract_pins.md — inert defaults on scan rows). Bruno: bruno/ podcasts/ mirrors the §8 endpoints (12 requests) + Bookhoard env vars. tests/ui/COVERAGE.md rows 18-23: podcasts pages/lifecycle/RLS/chain — Playwright sweep pending. |
||
|
John O'Keefe
|
45b1152a15 |
feat(podcasts): API endpoints — add/list/detail, subscriptions, downloads, sequence, search (task 1.7)
The §8 surface, wired through main.go → router.Config → /api/podcasts (JWT). Scoping is the engine's (D20): lookups outside the caller's scope simply return no rows → 404; the only Go-level checks are explicit-argument authorization (AddFeed's library + allow_member_feeds gate) and the admin role on feed deletion. Endpoints: POST/GET /feeds (add: fetch+parse+upsert+subscribe, optional download_all; list: subscriptions + episode counts + facet values computed in Go — feed counts are tens), GET /feeds/:id (detail with download states + gap count), PUT/DELETE /subscription (the ONLY user-writable podcast state, D9; unsubscribe retains files per D6), POST /refresh, POST /episodes/:eid/download (+ episode_ids[] bulk and eid='all' = Download All, exactly the non-downloaded set), PUT /order (D5 per-user override in the reader_settings KV — podcast_order:<feed_id>; 'native' deletes the row so default and explicit choice are indistinguishable), GET /sequence (effective order — override applied, native fallback — DOWNLOADED-only with the gap list; this payload powers web chain mode and, per §15, the future media3 queue), GET /search (subscription-scoped feed + episode search covering un-downloaded and historical rows, D14), DELETE /feeds/:id (admin; best-effort file removal, cascade deletes rows). Gap definition (countGaps): an un-downloaded episode with a downloaded episode AFTER it in list order — the D8 'middle episodes missing' banner condition. |
||
|
John O'Keefe
|
4d697f5fcf |
test(visibility): full suite green under RLS — sync paths, device-auth bridge, seed pool
Phase 5 (plan §7): the entire integration suite passes with the RLS policies live. The suite was the triage enumerator the plan predicted: - device_auth middleware: the THIRD identity bridge — KOReader/Kobo device-authenticated requests read media as the DEVICE OWNER (plan A9 called this path out; the flip was 404s on sync metadata). - conflicts/analytics/progress handlers: detached context.Background() reads threaded to the request context (the requesting user's identity) — 12+ sites; resolve helpers take ctx. - scanner: ScanFolders self-declares its configured admin identity (server maintenance, product-correct for any caller); generic job creation stamps the requesting admin. - harness: setup.DBPool and the sync-test pool declare rls.SeedIdentity at SESSION scope via BeforeAcquire — the pool never serves a second identity, so no A3 bleed; setup.DB falls back to the seed identity (NewSeedDB) for ungoverned-context calls; filter tests use seedExec for raw governed-table statements. - contract pin WIDENED (docs/contract_pins.md 2026-10-29): under RLS, item-scoped gates may answer 404 (row invisible — no existence leak) as well as 403 (Go-level checks on listings keep strict 403). - TestWorker_JobStatusTracking — the pre-existing red from the Phase 0 baseline — passes with the harness seed work; no product change. |
||
|
John O'Keefe
|
bb2cc26e9d |
feat(scanner): scanner entrypoints + generic jobs carry RLS identity; SeedDB harness fallback
The full suite enumerated the remaining triage class: tests drive the scanner directly (ScanFolders(context.Background())) and read seeded media via Queries with bare contexts — every governed read raised. - MediaScanner.ScanFolders self-declares its configured admin (SetAdminID — set by main AND the test helpers): a scan is server maintenance, product-correct regardless of caller. - ScopedDB gains a Fallback identity, exposed as NewSeedDB for the TEST HARNESS ONLY: setup.DB falls back to rls.SeedIdentity when the context carries none, so direct seeding/verification helpers work uniformly instead of per-site context threading. rls.SeedIdentity is the shared constant (no library_visibility rows => full visibility). - generic job creation (jobs.go) stamps the requesting admin into the job Context, same as the scan endpoint. The sync queue processor stays identity-free: sync_queue is not an RLS-governed table. |
||
|
John O'Keefe
|
922b92a8b6 |
feat(db): RLS library-visibility enforcement — invariant GREEN
The dashboard leak is closed at the database engine. The black-box
invariant (restricted user + hidden library => absent from every
media-returning surface) went RED->GREEN with this change set; clause-
protected surfaces never regressed, proving enforcement came from the
policies, not the test.
Phase 3 of plan ~/.opencode/plan/server-rls-library-visibility.md:
- schema.sql: idempotent RLS block — ENABLE + FORCE on media_items,
SELECT policy mirroring COALESCE(is_visible,true) exactly, permissive
per-command write stand-ins (FOR ALL would OR into SELECT).
- internal/rls + internal/database/scoped.go: ScopedDB declares the
acting user per statement (LOCAL GUC). DESIGN CORRECTION during
execution: the decorator never refuses — the ENGINE is the fail-loud
layer (policies raise on governed tables; auth lookups and settings,
ungoverned, simply work). A decorator refusal would have been a new
convention gate — plan B thinking — and broke login (chicken-and-egg:
credentials are checked before any identity exists).
- database/schema.go: the migration tx declares a synthetic identity —
schema.sql's dedup DO block SELECTs media_items, which raises once
policies are live (plan A11). Caught by the two-boot gate: boot 1
created the policies after the block ran; boot 2 raised.
- router.go: RLS bridge at BOTH c.Set("user") sites (JWT + device
auth) — echo-local c.Set never reaches the stdlib ctx the DB layer
receives.
- search: the search service's raw DBTX path receives the ScopedDB
itself, never the bare pool.
- harness: the server under test is wired exactly like main.go; the
seed handle is also ScopedDB-wired but carries seedCtx() — a
synthetic identity with no visibility rows — because sqlc :one
inserts use RETURNING, which evaluates the SELECT policy even on
INSERT. Regex UUID-scraping in the invariant replaced with a
structured JSON decode.
- invariant test: RED -> GREEN across every surface incl. View All;
403 pins and admin-scoping pins hold.
|
||
|
John O'Keefe
|
8c3d919aa0 |
wip(rls): engine-side fail-loud + harness seed identity [pin: work interrupted]
Design correction discovered by the auth chicken-and-egg: the decorator-level refusal was re-created a convention gate (plan B thinking). ScopedDB now NEVER refuses — it declares the GUC when the context carries an identity and declares nothing otherwise; the ENGINE is the fail-loud layer (policies raise on governed tables, ungoverned tables like auth lookups simply work). rls.WithInternal removed as a dead concept. - search service's raw DBTX path now receives the ScopedDB itself (main.go + harness) — a bare pool bypassed the GUC. - database/schema.go: migration tx declares a synthetic identity (schema.sql's dedup DO block SELECTs media_items; with policies live, boot 2 raised 'unrecognized configuration parameter' — A11). - harness: TestServerSetup.DB is a raw-pool SEED handle (write policies permissive, other seeded tables ungoverned); the server under test keeps the scoped wiring. 33 CreateMediaItem sites use seedCtx() — a seed identity with no visibility rows — because RETURNING requires SELECT visibility of the new row. - invariant test: regex UUID-scraping replaced with a structured JSON decode of the sections payload. INTERRUPTION NOTE: an edit to this same effect (seedCtx) was applied, confirmed by the tool, and subsequently vanished from disk while an 'opencode serve' process (PID 84551) is alive on the machine. This commit pins the tree against further loss; implementation is PAUSED pending the user killing that process. |
||
|
John O'Keefe
|
004d4197a4 | fix(test): rls fixture — reuse declared err variable | ||
|
John O'Keefe
|
f4b581872f |
wip(rls): ScopedDB + rls package + schema policies + echo bridge [pin compiling state]
Phase 3 of the RLS plan, pinned mid-flight because a parallel writer
was actively editing cmd/server/tests/rls_visibility_test.go (A14):
the file flip-flopped between two implementations and gained a compile
error after the previous commit. This snapshot makes the whole tree
compile; functional verification (two-boot gate, invariant green,
full suite) lands in the next commit(s).
- internal/rls: typed ctx keys — user identity (bridged from echo by
the router) and an INTERNAL marker whose semantics grant nothing:
no GUC is declared, so policy'd tables still raise at the engine;
only RLS-ungoverned tables (system_settings, users counts) work for
user-less startup paths (settings registry Load, CountAdmins).
- internal/database/scoped.go: DBTX decorator — per-statement tx,
commit-on-Close (:many) / commit-on-Scan (:one), error paths always
terminate the tx, refusal before SQL when no identity and not
internal.
- database/schema/schema.sql: idempotent policy block (DROP IF EXISTS
+ CREATE — schema.sql re-runs every boot): ENABLE + FORCE, read
policy exactly mirroring COALESCE(is_visible,true), permissive
per-command write stand-ins (FOR ALL would OR into SELECT).
- router.go: RLS bridge at BOTH c.Set("user") sites (JWT + device
auth) — echo-local storage never reaches the stdlib ctx.
- main.go + test harness: database.New over ScopedDB at both wiring
sites; startup reads use rls.WithInternal.
|
||
|
John O'Keefe
|
a19d7a4cbf |
test(visibility): fixture fixes from first live runs
Three harness corrections found while executing the invariant test: - library type is 'ebooks' (plural) in library_types, not 'ebook' - cleanup ordering: t.Cleanup is LIFO — register the pool Close BEFORE the fixture's library deletes, or the deletes run against a closed pool and fail silently into ignored errors (leaving RLSMARKER rows behind; found because a second run saw the first run's 'deleted' books still leaking through Recently Added) - collection detail is addressed by the section's collection_id (the UUID in view_all_url), not the section name |
||
|
John O'Keefe
|
3dcb794032 |
test(visibility): black-box invariant — hidden library absent from every surface
Adds the permanent auditor for per-user library visibility: a restricted user with a hidden library sees none of its items on any media-returning surface (media list, search, dashboard sections, series, facets, OPDS device catalog, collection detail, scoped/detail access). RED now, on exactly the surfaces the audit predicted: the dashboard section queries (GetNotStartedItems / GetRecentlyAddedItems / GetCollectionItemsForDashboard et al.) never carried the visible_library_ids clause, and the collection-detail endpoint (mobile View All) exposes the full section query results. Clause-protected surfaces (media list, search, series) pass, proving the test itself isn't a false positive. Implementation-agnostic on purpose: it asserts the user-level invariant through the public API, so it stays valid when RLS replaces the per-query clauses and catches any future regression regardless of how the code is written. Also pins: hidden library_id=403 (never silently passed), item detail denial, and that the hide is USER-scoped (admin still sees the hidden library). Test-lesson encoded: cleanup ordering is LIFO — the pool Close must be registered BEFORE fixture cleanups, or library deletes run against a closed pool and fail silently into ignored errors. |
||
|
John O'Keefe
|
795225f097 |
fix(scanner): reconcile archive pass per library, not per folder
Release / build-and-push (push) Successful in 3m5s
The archive pass looped over the scan's folders and, inside each
iteration, compared the ENTIRE library's item list against that one
folder's walk. With two registered folders every item living under the
other folder was flagged "missing (1/2)" on every reconciling scan —
prod's audiobook library (the only multi-folder one) showed 743/743
flagged — and nothing reached the two-consecutive-miss threshold only
because each scan's walk clears missing_scan_count before the pass
re-flags it. With three or more folders the defect stops bouncing: a
later iteration sees an earlier iteration's count-1 marks on items its
own walk lacks and archives them mid-pass — one registered folder away
from archiving a third of a library per scan. Single-folder libraries
were unaffected, which is why comics/ebooks/manga showed zero flags.
Folder-book presence was suspected alongside it (audioBookRoots read
during the comparison), but the map is written for every >=2-track
folder-book root on every walk (processAudioBookFile, since
|
||
|
John O'Keefe
|
fd9292e6f2 |
fix(series): Date Added respects the direction toggle; bump bundle stamp
Two follow-ups from live testing: - "recent" is BOTH a base toolbar key and a full registry key (where it means desc). ResolveSeriesSort's full-key passthrough ran first, so sort=recent&dir=asc silently resolved back to desc — the direction chevron flipped but Date Added could never sort ascending. "recent" now composes with the direction before the passthrough, unknown bases fall to the default entirely (a hostile dir can't ride an unknown sort), and the composition table is unit-pinned plus an HTTP-level row in the series filter test. - The header search "still broken" report was a stale-cache artifact: the fixed code shipped but main.js kept its ?v=7 stamp, so browsers with a cached copy never refetched (fresh-profile e2e passed while the user's browser didn't — heuristically cached module). Stamp bumped to ?v=8 in header + docs; the e2e suite re-verified against the rebuilt server. |
||
|
John O'Keefe
|
fb1645c90c |
fix(metadata): m4b text-track fallback; in-app confirm; save-first write
Three defects from real-world use: - The m4b rewrite assumed every stream survives -map 0 -c copy; files carrying text-ish tracks (the movenc 'text' fourcc) fail the ipod muxer with "Tag text incompatible with output codec id" — exactly the error a write attempt hit in the field. The rewrite now attempts full fidelity first, then retries mapping audio + attached pictures only (-dn -sn): chapters live in the container header (-map_chapters 0) so they survive; every attempt still ffprobe-validates before the atomic rename and the original is never touched on total failure. - The browser confirm() is replaced by an in-app dialog (WriteToFileConfirmDialog, Alpine + templ — no native dialogs). - "Write to file" now SAVES the pending editor edits first (persisted through the same PUT the Save button uses), then writes the file from those DB values, and closes the editor + refreshes the page exactly like Save — previously the endpoint wrote the LAST-SAVED DB values, so edits sitting in the modal never reached the file, and the page didn't visibly update. Test coverage per the owner's bar: EVERY single-file container (MP3, FLAC, M4A, M4B) gets its own runtime-generated fixture, gets written, and the FILE's metadata is then verified independently — ffprobe reads the rewritten container and every mapped key is asserted per format, including the subtitle round-trip split (MP3/FLAC yes, M4A/B no) — and the EPUB/CBZ/sidecar assertions now PARSE the written XML (dc titles/ creators, ComicInfo fields, calibre meta) instead of substring matching. A smart-genre unit test also pins the mixed-blob contract: a recognized shelf wins the genre column even when demoted format words ride along in the same blob. |
||
|
John O'Keefe
|
96c1228805 |
feat(series): multi-field search, sort-direction toggle, apply placement
Toolbar feedback round: the series search box now matches whatever the main search matches for a group — series NAME, any book's AUTHOR, and any book's TITLE inside the series (WHERE EXISTS per group-row, so counts stay true); the separate author input is gone from the toolbar (the author= param stays live for the JSON API and deep links). The sort control becomes a base-key select (Date added/Name/Author/Books/ Progress/Last read) plus a chevron direction toggle that flips and resubmits; sort/dir compose through ResolveSeriesSort (full registry keys keep working for back-compat, unknown base keys fall to recent). The search/apply button moved next to the search box as requested, and default URLs stay clean (sort/dir only emitted once the user chooses). |
||
|
John O'Keefe
|
b462c7fb44 |
test(scanner): pin the demoted-genre contract the genre fix intended
TestAudiobookTagFallbacks still asserted the verbatim "Audiobook" genre
column, but
|
||
|
John O'Keefe
|
59eddc1386 |
feat(metadata): audio tag rewrite via ffmpeg + CBZ ComicInfo injection
Phases 5b and 5c of the metadata writer. 5b — single-file audio (MP3/FLAC/M4A/M4B): the container's tags are rewritten with ffmpeg -c copy (no re-encode; -map 0 keeps cover art, -map_chapters 0 keeps chapters). The key set is exactly what the scanner's applyAudioTagFallbacks reads back — title+album, subtitle, artist/album_artist, composer (narrators, Audible's convention), grouping + iTunes series-part, date, genre — so a write + rescan round-trips. Output is ffprobe-VALIDATED (duration within 1s/0.5% of the source) before the atomic rename; a lossy container rewrite never replaces a good book. Fields the format can't carry per our own read chain are deliberately not written, and fields_written reports only what really round-trips (subtitle rides MP3/FLAC — this image's mp4 muxer drops subtitle keys, discovered empirically and pinned by test). 5c — CBZ: ComicInfo.xml injected (or replaced) with the same atomic zip rewrite as the EPUB path, extended with an append-if-missing mode since a comic may not carry one yet. Field map mirrors the scanner's comic block: Title/Series/Number/Count/Volume, Writer (authors), Summary, Publisher, Year/Month/Day, LanguageISO, Web, Genre+Tags; fractional series numbers are omitted rather than truncated to a lie. CBR/CB7 stay on the sidecar path (RAR/7z can't be rewritten in place). Integration tests generate media at test runtime with ffmpeg (m4b) and in-memory zips (cbz) — no binary fixtures — pinning the tag round-trip through ffprobe, post-write rescan titles, the ComicInfo injection and the validation-gate behavior. |
||
|
John O'Keefe
|
eaf1b2c8dd |
feat(metadata): write-to-file — embedded EPUB OPF + Calibre sidecars
Phase 5a of the metadata-writer feature: the book editor gains a "Write to file" action (admin) that persists the item's CURRENT database metadata into the artifact on disk, so the library stays portable to Calibre and other apps — the thing Save alone can't do. Dispatch (services/metadata_writer.go): EPUB/KEPUB gets its embedded OPF spliced in place (fresh dc:* block generated inside the ORIGINAL metadata open tag; manifest/spine/guide untouched; every other zip entry copied byte-identical in original order so mimetype stays first and STORED; original dc:identifiers preserved — the package's unique-identifier must keep resolving). Everything else takes a Calibre-style sidecar: metadata.opf when the book is ALONE in its directory (Calibre's own convention — round-trips through Calibre too), <basename>.opf when siblings share the folder, so one book's sidecar can never poison dir-mates' scans (the scanner now reads both, basename first). Safety contract: temp file + atomic rename — the original is never truncated on failure; the generated OPF is unit-tested to ROUND-TRIP through the scanner's own parser (write side and read side can never disagree); metadata_overrides are deliberately kept (file and DB agree after the write; overrides still guard against stale sidecars). The endpoint re-extracts inline (RescanMediaItem) and reports rescan failure separately — a failed rescan doesn't undo the write. SHA-256 changes are inherent and documented for users (Kobo/KOReader re-key). UI: the metadata editor modal carries the file path as a data attribute; the button confirms before mutating, labels the target (embedded vs sidecar) and toasts the outcome. Integration tests pin the 403, missing-file error, EPUB embed + post-write rescan title, the lone-book metadata.opf case, and the shared-dir <basename>.opf case with no dir-scoped litter. |
||
|
John O'Keefe
|
61e5f47ec0 |
feat(search): viewer progress + completed badges; dropdown to templ
Search results now show where you are in a book: a completed check at the 0.999 threshold (same cut as the filter status arms), a partial progress bar along the cover while mid-read, nothing when you never opened it. Rendered in BookCard (any grid whose adapter loads viewer progress) and in the header-search dropdown rows. The progress rides a LEFT JOIN reading_progress on the SAME user_id param SearchMediaItemsUnified already binds for library visibility — one requester param, no new wire inputs, row-neutral join (the table is UNIQUE per media_item+user). SearchMediaItemsUnifiedRow and BookInfo gain the keys additively (viewer_percentage null when unknown, library_name empty on feeds that don't load it); contract_pins.md carries the changelog entry and the pin tests widen in lockstep. The header dropdown itself migrates off the innerHTML string renderer in search.ts (a rendering-ownership violation: a templ twin existed via the HX-Request branch). The input now hx-gets layout=dropdown and swaps the new templ SearchResults partial — titles/authors highlighted server-side (escape-first, safe by construction), rows with the same progress overlays — while a slim Alpine component owns only client-local state (min-length gating, open/close, keyboard selection, htmx lifecycle). web/src/rendering-allowlist.txt drops the search.ts baseline to zero accordingly. Integration test pins the per-user join (fresh user sees 1.0/0.42/null on the fixture; the shared admin sees null everywhere — read-only) and the dropdown partial's overlays/count/row count. |