Reproduced the §0 item 2 suspect: echo v5 bool-binds form values via
strconv.ParseBool, which accepts "true"/"1" but REJECTS the
browser-native <input type=checkbox> value "on" — and a bind failure
kills the WHOLE AddFeed request (400 "invalid request"), not just
the checkbox. The add modal posts urlencoded (htmx), so any
checkbox-serialization that emits "on" made Download All silently
unusable.
Fix: FlexBool (encoding.TextUnmarshaler-style UnmarshalParam) binds
download_all permissively on every wire — "true"/"on"/"yes"/"checked"
=> true, explicit false/0/off => false, ABSENCE (unchecked box on any
wire) => false; JSON bool binding is native and unaffected. Tolerates
the exact wires htmx and browsers emit.
AddFeed's subscription flip now logs "[PODCAST] flip ... <feed> —
user" so the live flip is provable in app logs (user suspected it
never fired).
Integration test TestPodcastDownloadAllFormEncoding pins the urlencoded
wire (what htmx actually sends): true + on both flip the caller's
policy to 'all', an unchecked add leaves the default.
Logout wiped localStorage but never cleared the token cookie (HttpOnly —
only a server response can) and never called the logout API, leaving the
refresh token live for 30 days and every cookie-authenticated SSR page
accessible: the fake-signout bug. Reproduced on prod and dev; unrelated
to RLS (it sits behind authentication) and long-standing.
- /api/auth/logout now ALWAYS clears the session cookie (Set-Cookie
empty + Max-Age=-1, HttpOnly/Path/SameSite mirroring login) — even a
malformed body must sign the browser out; refresh-token revocation
unchanged.
- The header logout button (and the rules-page copy) now POST to the
API with the stored refresh token BEFORE clearing local state and
redirecting — best effort: local cleanup never blocks on API failure.
- toast 401 handler fixed: it cleared 'refreshToken' but storage uses
'refresh_token' — the real key.
- Regression test TestLogoutClearsSession: real login → logout →
response deletes the cookie; logged-out refresh token can't mint
access tokens; body-less logout still clears.
Also carries regenerated artifacts from the confirm-modal and
episode-list work (header_templ/podcast_detail_templ/style.css).
WHY: downloads needed manual refreshes to see progress; the 204 +
HX-Trigger event indirection missed its listener, and the worker
buffered whole enclosures so no progress existed to show.
- Worker streams enclosures to disk through a counting loop (64KiB
chunks, scheme/redirect/size guards preserved) with an in-memory
per-episode progress snapshot (received/total; total from
Content-Length, seeded by the feed's declared length). Deliberately
transient — the DB row state remains the durable truth.
- Episode-list renderer seam: the router injects a closure into
PodcastHandler (handlers can't import templates — MediaDetail cycle)
so htmx download/retry/Download All/refresh/order/policy actions
answer with the REFRESHED partial directly (200, hx-target=
#episode-list) — no events to miss. Non-HX callers keep JSON; tests
without a renderer keep 204+trigger.
- Polling tightens to 2s while downloads are active; downloading rows
render a percentage bar + MB counter (quote-free inline style per
templ gotcha #8) or an indeterminate bar when the server sends no
length.
- Issue #3: Download All (button AND the add-feed checkbox) flips the
CALLER's subscription policy to 'all' — per-user, never global (D6:
files shared, policies not) — and the auto-download select
out-of-band-swaps so the dropdown shows what the user is getting.
They can turn it down afterwards; the flip is honest communication,
not coercion (user-approved 2026-10-29).
PINs: Download All with HX answers 200 + episode-list + oob select and
flips newest → all; add-feed download_all sets policy all.
WHY: the /podcasts search box filtered feed CARDS only (title, display
name, host, publisher, feed description). D14's episode search existed
only as the /api/podcasts/search endpoint and was never wired into the
page — searching 'Quarter' (present only in episode titles) cleared the
grid while 'Trader' (feed metadata) survived.
The page query now consults SearchMyPodcastEpisodes (subscription-scoped
via RLS — the caller's identity) when a query is present; a feed
surfaces when its OWN metadata OR any of its episodes matches, with an
'N matching episode(s)' badge on the card.
PIN: SSR test with a needle that exists only in an episode title — the
feed must surface with the match count.
WHY: htmx sends non-GET requests urlencoded by default; the SetFeedOrder
request struct only carried a json tag, so the binder left Order empty
and 'required' fired — the exact error the order dropdown AND the
Serial-suggestion button produced (hx-vals JSON also arrives
form-encoded). One form tag fixes both controls.
Regression test drives the exact wire format: urlencoded PUT with
order=oldest/native → 200, empty/invalid → 400.
Invariant 3: progress anchors on the episode row (guid identity +
chapter/offset), never the ordinal. Renumbering feed_position (the
fetcher's per-fetch renumber case) leaves the progress row untouched
and the sequence membership stable — downloaded-only with no gaps.
cmd/server/tests/podcast_test.go — six suites, the shared-dev-DB
doctrine (test*-named libraries, cascading cleanup, no binary
fixtures):
- TestPodcastRLSSurfaces: THE AUDITOR (invariant 13) — non-subscriber
404 on feed detail + SSR page (existence leak), episodes absent from
generic media reads, admin escape, subscriber visibility, then the
hidden-library flip hiding an already-subscribed feed (the feed
policy's library-visibility join). Seeding runs as the FIRST ADMIN —
the seed identity has neither subscription nor users row, so podcast
INSERT..RETURNING (SELECT policy under FORCE) 42501s for it;
production pods run as exactly that admin (invariant 12). All direct
row probes read through the admin scope for the same reason.
- TestPodcastAddFeedFlow: member add + subscribe + metadata-only
upsert; duplicate-URL dedupe (no second row); allow_member_feeds
gate (member 403 / admin 201 / subscribe-still-allowed).
- TestPodcastShrunkenFeedRetention: feed purges an episode from the
XML, both rows persist (upsert-without-delete pin, D14).
- TestPodcastEditSegregation: generic edit 400 + write-to-file refusal
for episodes; control book edit unaffected.
- TestPodcastDownloadWorker (ffmpeg-generated tone at runtime): file
lands at <download-root>/<feed-slug>/NNNN-…, audio_metadata probed;
re-download of a downloaded episode never re-fetches (idempotency
pin, enclosure-hit counted); dead enclosure → failed + persisted
reason + metadata intact. Fetch/SSRF tests swap the client via
SetPodcastHTTPClientForTests (the production client refuses
loopback httptest servers by design — the real client gets its own
negative test in services).
- TestPodcastScannerExclusion: SetFolders on a podcasts library root
yields zero scanner folders (invariant 4 pin) — MediaScanner
.FolderCount() accessor added.
Unit: internal/services/podcast_rss_test.go — namespace-aware parsing
fixture (three XML namespaces, fallback chains, hosts/guests),
duration spellings, RFC-822 variants, explicit variants, slug/path
layout, enclosure-extension selection, stripHTML, and the SSRF matrix
(private/loopback/link-local/multicast/mapped-v6 blocked; a loopback
httptest server PROVES the production client refuses it).
suites green: full integration run 161s / 0 failures; vitest 12 files
163 tests.
Contract widening: TestContractPin_MediaListBothBranches' raw-row
allowlist gains the 15 podcast episode columns (additive; documented in
docs/contract_pins.md — inert defaults on scan rows). Bruno: bruno/
podcasts/ mirrors the §8 endpoints (12 requests) + Bookhoard env vars.
tests/ui/COVERAGE.md rows 18-23: podcasts pages/lifecycle/RLS/chain —
Playwright sweep pending.
Phase 5 (plan §7): the entire integration suite passes with the RLS
policies live. The suite was the triage enumerator the plan predicted:
- device_auth middleware: the THIRD identity bridge — KOReader/Kobo
device-authenticated requests read media as the DEVICE OWNER
(plan A9 called this path out; the flip was 404s on sync metadata).
- conflicts/analytics/progress handlers: detached
context.Background() reads threaded to the request context (the
requesting user's identity) — 12+ sites; resolve helpers take ctx.
- scanner: ScanFolders self-declares its configured admin identity
(server maintenance, product-correct for any caller); generic job
creation stamps the requesting admin.
- harness: setup.DBPool and the sync-test pool declare
rls.SeedIdentity at SESSION scope via BeforeAcquire — the pool never
serves a second identity, so no A3 bleed; setup.DB falls back to the
seed identity (NewSeedDB) for ungoverned-context calls; filter tests
use seedExec for raw governed-table statements.
- contract pin WIDENED (docs/contract_pins.md 2026-10-29): under RLS,
item-scoped gates may answer 404 (row invisible — no existence leak)
as well as 403 (Go-level checks on listings keep strict 403).
- TestWorker_JobStatusTracking — the pre-existing red from the Phase 0
baseline — passes with the harness seed work; no product change.
The full suite enumerated the remaining triage class: tests drive the
scanner directly (ScanFolders(context.Background())) and read seeded
media via Queries with bare contexts — every governed read raised.
- MediaScanner.ScanFolders self-declares its configured admin
(SetAdminID — set by main AND the test helpers): a scan is server
maintenance, product-correct regardless of caller.
- ScopedDB gains a Fallback identity, exposed as NewSeedDB for the
TEST HARNESS ONLY: setup.DB falls back to rls.SeedIdentity when the
context carries none, so direct seeding/verification helpers work
uniformly instead of per-site context threading. rls.SeedIdentity is
the shared constant (no library_visibility rows => full visibility).
- generic job creation (jobs.go) stamps the requesting admin into the
job Context, same as the scan endpoint.
The sync queue processor stays identity-free: sync_queue is not an
RLS-governed table.
The dashboard leak is closed at the database engine. The black-box
invariant (restricted user + hidden library => absent from every
media-returning surface) went RED->GREEN with this change set; clause-
protected surfaces never regressed, proving enforcement came from the
policies, not the test.
Phase 3 of plan ~/.opencode/plan/server-rls-library-visibility.md:
- schema.sql: idempotent RLS block — ENABLE + FORCE on media_items,
SELECT policy mirroring COALESCE(is_visible,true) exactly, permissive
per-command write stand-ins (FOR ALL would OR into SELECT).
- internal/rls + internal/database/scoped.go: ScopedDB declares the
acting user per statement (LOCAL GUC). DESIGN CORRECTION during
execution: the decorator never refuses — the ENGINE is the fail-loud
layer (policies raise on governed tables; auth lookups and settings,
ungoverned, simply work). A decorator refusal would have been a new
convention gate — plan B thinking — and broke login (chicken-and-egg:
credentials are checked before any identity exists).
- database/schema.go: the migration tx declares a synthetic identity —
schema.sql's dedup DO block SELECTs media_items, which raises once
policies are live (plan A11). Caught by the two-boot gate: boot 1
created the policies after the block ran; boot 2 raised.
- router.go: RLS bridge at BOTH c.Set("user") sites (JWT + device
auth) — echo-local c.Set never reaches the stdlib ctx the DB layer
receives.
- search: the search service's raw DBTX path receives the ScopedDB
itself, never the bare pool.
- harness: the server under test is wired exactly like main.go; the
seed handle is also ScopedDB-wired but carries seedCtx() — a
synthetic identity with no visibility rows — because sqlc :one
inserts use RETURNING, which evaluates the SELECT policy even on
INSERT. Regex UUID-scraping in the invariant replaced with a
structured JSON decode.
- invariant test: RED -> GREEN across every surface incl. View All;
403 pins and admin-scoping pins hold.
Design correction discovered by the auth chicken-and-egg: the
decorator-level refusal was re-created a convention gate (plan B
thinking). ScopedDB now NEVER refuses — it declares the GUC when the
context carries an identity and declares nothing otherwise; the ENGINE
is the fail-loud layer (policies raise on governed tables, ungoverned
tables like auth lookups simply work). rls.WithInternal removed as a
dead concept.
- search service's raw DBTX path now receives the ScopedDB itself
(main.go + harness) — a bare pool bypassed the GUC.
- database/schema.go: migration tx declares a synthetic identity
(schema.sql's dedup DO block SELECTs media_items; with policies
live, boot 2 raised 'unrecognized configuration parameter' — A11).
- harness: TestServerSetup.DB is a raw-pool SEED handle (write
policies permissive, other seeded tables ungoverned); the server
under test keeps the scoped wiring. 33 CreateMediaItem sites use
seedCtx() — a seed identity with no visibility rows — because
RETURNING requires SELECT visibility of the new row.
- invariant test: regex UUID-scraping replaced with a structured JSON
decode of the sections payload.
INTERRUPTION NOTE: an edit to this same effect (seedCtx) was applied,
confirmed by the tool, and subsequently vanished from disk while an
'opencode serve' process (PID 84551) is alive on the machine. This
commit pins the tree against further loss; implementation is PAUSED
pending the user killing that process.
Phase 3 of the RLS plan, pinned mid-flight because a parallel writer
was actively editing cmd/server/tests/rls_visibility_test.go (A14):
the file flip-flopped between two implementations and gained a compile
error after the previous commit. This snapshot makes the whole tree
compile; functional verification (two-boot gate, invariant green,
full suite) lands in the next commit(s).
- internal/rls: typed ctx keys — user identity (bridged from echo by
the router) and an INTERNAL marker whose semantics grant nothing:
no GUC is declared, so policy'd tables still raise at the engine;
only RLS-ungoverned tables (system_settings, users counts) work for
user-less startup paths (settings registry Load, CountAdmins).
- internal/database/scoped.go: DBTX decorator — per-statement tx,
commit-on-Close (:many) / commit-on-Scan (:one), error paths always
terminate the tx, refusal before SQL when no identity and not
internal.
- database/schema/schema.sql: idempotent policy block (DROP IF EXISTS
+ CREATE — schema.sql re-runs every boot): ENABLE + FORCE, read
policy exactly mirroring COALESCE(is_visible,true), permissive
per-command write stand-ins (FOR ALL would OR into SELECT).
- router.go: RLS bridge at BOTH c.Set("user") sites (JWT + device
auth) — echo-local storage never reaches the stdlib ctx.
- main.go + test harness: database.New over ScopedDB at both wiring
sites; startup reads use rls.WithInternal.
Three harness corrections found while executing the invariant test:
- library type is 'ebooks' (plural) in library_types, not 'ebook'
- cleanup ordering: t.Cleanup is LIFO — register the pool Close BEFORE
the fixture's library deletes, or the deletes run against a closed
pool and fail silently into ignored errors (leaving RLSMARKER rows
behind; found because a second run saw the first run's 'deleted'
books still leaking through Recently Added)
- collection detail is addressed by the section's collection_id (the
UUID in view_all_url), not the section name
Adds the permanent auditor for per-user library visibility: a restricted
user with a hidden library sees none of its items on any media-returning
surface (media list, search, dashboard sections, series, facets, OPDS
device catalog, collection detail, scoped/detail access).
RED now, on exactly the surfaces the audit predicted: the dashboard
section queries (GetNotStartedItems / GetRecentlyAddedItems /
GetCollectionItemsForDashboard et al.) never carried the
visible_library_ids clause, and the collection-detail endpoint (mobile
View All) exposes the full section query results. Clause-protected
surfaces (media list, search, series) pass, proving the test itself
isn't a false positive.
Implementation-agnostic on purpose: it asserts the user-level invariant
through the public API, so it stays valid when RLS replaces the
per-query clauses and catches any future regression regardless of how
the code is written. Also pins: hidden library_id=403 (never silently
passed), item detail denial, and that the hide is USER-scoped (admin
still sees the hidden library).
Test-lesson encoded: cleanup ordering is LIFO — the pool Close must be
registered BEFORE fixture cleanups, or library deletes run against a
closed pool and fail silently into ignored errors.
The archive pass looped over the scan's folders and, inside each
iteration, compared the ENTIRE library's item list against that one
folder's walk. With two registered folders every item living under the
other folder was flagged "missing (1/2)" on every reconciling scan —
prod's audiobook library (the only multi-folder one) showed 743/743
flagged — and nothing reached the two-consecutive-miss threshold only
because each scan's walk clears missing_scan_count before the pass
re-flags it. With three or more folders the defect stops bouncing: a
later iteration sees an earlier iteration's count-1 marks on items its
own walk lacks and archives them mid-pass — one registered folder away
from archiving a third of a library per scan. Single-folder libraries
were unaffected, which is why comics/ebooks/manga showed zero flags.
Folder-book presence was suspected alongside it (audioBookRoots read
during the comparison), but the map is written for every >=2-track
folder-book root on every walk (processAudioBookFile, since 6216bd9b);
the flagged folder-books were victims of the same cross-folder
comparison. The roots attribution did carry the same unchecked
strings.HasPrefix boundary bug as the rest of the scanner (8th site,
variable named `root`, missed by the earlier sweep); it now uses
folderContains.
The pass now groups folders by owning library and, per library,
compares ListMediaItemsByLibraryIncludingArchived against the UNION of
all that library's folders' walks. The coverage (partial-scope) guard,
the walk-failure skip, and the mark/archive/purge item loop are
unchanged.
Regression test (archive_multifolder_test.go) builds a two-folder
audiobooks library — single-file book in one folder, folder-of-tracks
book in the other — and scans twice: pre-fix both items are flagged
missing on both scans (verified by running the test against the old
code), post-fix neither is ever flagged or archived.
Two follow-ups from live testing:
- "recent" is BOTH a base toolbar key and a full registry key (where it
means desc). ResolveSeriesSort's full-key passthrough ran first, so
sort=recent&dir=asc silently resolved back to desc — the direction
chevron flipped but Date Added could never sort ascending. "recent"
now composes with the direction before the passthrough, unknown bases
fall to the default entirely (a hostile dir can't ride an unknown
sort), and the composition table is unit-pinned plus an HTTP-level
row in the series filter test.
- The header search "still broken" report was a stale-cache artifact:
the fixed code shipped but main.js kept its ?v=7 stamp, so browsers
with a cached copy never refetched (fresh-profile e2e passed while
the user's browser didn't — heuristically cached module). Stamp bumped
to ?v=8 in header + docs; the e2e suite re-verified against the
rebuilt server.
Three defects from real-world use:
- The m4b rewrite assumed every stream survives -map 0 -c copy; files
carrying text-ish tracks (the movenc 'text' fourcc) fail the ipod
muxer with "Tag text incompatible with output codec id" — exactly the
error a write attempt hit in the field. The rewrite now attempts full
fidelity first, then retries mapping audio + attached pictures only
(-dn -sn): chapters live in the container header (-map_chapters 0)
so they survive; every attempt still ffprobe-validates before the
atomic rename and the original is never touched on total failure.
- The browser confirm() is replaced by an in-app dialog
(WriteToFileConfirmDialog, Alpine + templ — no native dialogs).
- "Write to file" now SAVES the pending editor edits first (persisted
through the same PUT the Save button uses), then writes the file
from those DB values, and closes the editor + refreshes the page
exactly like Save — previously the endpoint wrote the LAST-SAVED DB
values, so edits sitting in the modal never reached the file, and
the page didn't visibly update.
Test coverage per the owner's bar: EVERY single-file container (MP3,
FLAC, M4A, M4B) gets its own runtime-generated fixture, gets written,
and the FILE's metadata is then verified independently — ffprobe reads
the rewritten container and every mapped key is asserted per format,
including the subtitle round-trip split (MP3/FLAC yes, M4A/B no) — and
the EPUB/CBZ/sidecar assertions now PARSE the written XML (dc titles/
creators, ComicInfo fields, calibre meta) instead of substring
matching. A smart-genre unit test also pins the mixed-blob contract: a
recognized shelf wins the genre column even when demoted format words
ride along in the same blob.
Toolbar feedback round: the series search box now matches whatever the
main search matches for a group — series NAME, any book's AUTHOR, and
any book's TITLE inside the series (WHERE EXISTS per group-row, so
counts stay true); the separate author input is gone from the toolbar
(the author= param stays live for the JSON API and deep links). The
sort control becomes a base-key select (Date added/Name/Author/Books/
Progress/Last read) plus a chevron direction toggle that flips and
resubmits; sort/dir compose through ResolveSeriesSort (full registry
keys keep working for back-compat, unknown base keys fall to recent).
The search/apply button moved next to the search box as requested, and
default URLs stay clean (sort/dir only emitted once the user chooses).
TestAudiobookTagFallbacks still asserted the verbatim "Audiobook" genre
column, but 885d6140 demoted format words to tags-only — and that
commit's message explicitly says the pinned expectation was updated to
the demoted contract. The code change landed; the test edit it
describes never did, so the full suite has been red since 13:21 today
on a test my changes never touched.
Pin the new contract: the genre column stays empty for demoted format
words and the raw candidate ("Audiobook") lands in the tags array —
exactly the tags-only behavior the fix's message documents. Full
integration suite green after this.
Phases 5b and 5c of the metadata writer.
5b — single-file audio (MP3/FLAC/M4A/M4B): the container's tags are
rewritten with ffmpeg -c copy (no re-encode; -map 0 keeps cover art,
-map_chapters 0 keeps chapters). The key set is exactly what the
scanner's applyAudioTagFallbacks reads back — title+album, subtitle,
artist/album_artist, composer (narrators, Audible's convention),
grouping + iTunes series-part, date, genre — so a write + rescan
round-trips. Output is ffprobe-VALIDATED (duration within 1s/0.5% of
the source) before the atomic rename; a lossy container rewrite never
replaces a good book. Fields the format can't carry per our own read
chain are deliberately not written, and fields_written reports only
what really round-trips (subtitle rides MP3/FLAC — this image's mp4
muxer drops subtitle keys, discovered empirically and pinned by test).
5c — CBZ: ComicInfo.xml injected (or replaced) with the same atomic zip
rewrite as the EPUB path, extended with an append-if-missing mode
since a comic may not carry one yet. Field map mirrors the scanner's
comic block: Title/Series/Number/Count/Volume, Writer (authors),
Summary, Publisher, Year/Month/Day, LanguageISO, Web, Genre+Tags;
fractional series numbers are omitted rather than truncated to a lie.
CBR/CB7 stay on the sidecar path (RAR/7z can't be rewritten in place).
Integration tests generate media at test runtime with ffmpeg (m4b) and
in-memory zips (cbz) — no binary fixtures — pinning the tag round-trip
through ffprobe, post-write rescan titles, the ComicInfo injection and
the validation-gate behavior.
Phase 5a of the metadata-writer feature: the book editor gains a
"Write to file" action (admin) that persists the item's CURRENT
database metadata into the artifact on disk, so the library stays
portable to Calibre and other apps — the thing Save alone can't do.
Dispatch (services/metadata_writer.go): EPUB/KEPUB gets its embedded
OPF spliced in place (fresh dc:* block generated inside the ORIGINAL
metadata open tag; manifest/spine/guide untouched; every other zip
entry copied byte-identical in original order so mimetype stays first
and STORED; original dc:identifiers preserved — the package's
unique-identifier must keep resolving). Everything else takes a
Calibre-style sidecar: metadata.opf when the book is ALONE in its
directory (Calibre's own convention — round-trips through Calibre
too), <basename>.opf when siblings share the folder, so one book's
sidecar can never poison dir-mates' scans (the scanner now reads both,
basename first).
Safety contract: temp file + atomic rename — the original is never
truncated on failure; the generated OPF is unit-tested to ROUND-TRIP
through the scanner's own parser (write side and read side can never
disagree); metadata_overrides are deliberately kept (file and DB agree
after the write; overrides still guard against stale sidecars). The
endpoint re-extracts inline (RescanMediaItem) and reports rescan
failure separately — a failed rescan doesn't undo the write. SHA-256
changes are inherent and documented for users (Kobo/KOReader re-key).
UI: the metadata editor modal carries the file path as a data
attribute; the button confirms before mutating, labels the target
(embedded vs sidecar) and toasts the outcome. Integration tests pin
the 403, missing-file error, EPUB embed + post-write rescan title, the
lone-book metadata.opf case, and the shared-dir <basename>.opf case
with no dir-scoped litter.
Search results now show where you are in a book: a completed check at
the 0.999 threshold (same cut as the filter status arms), a partial
progress bar along the cover while mid-read, nothing when you never
opened it. Rendered in BookCard (any grid whose adapter loads viewer
progress) and in the header-search dropdown rows.
The progress rides a LEFT JOIN reading_progress on the SAME user_id
param SearchMediaItemsUnified already binds for library visibility —
one requester param, no new wire inputs, row-neutral join (the table
is UNIQUE per media_item+user). SearchMediaItemsUnifiedRow and BookInfo
gain the keys additively (viewer_percentage null when unknown,
library_name empty on feeds that don't load it); contract_pins.md
carries the changelog entry and the pin tests widen in lockstep.
The header dropdown itself migrates off the innerHTML string renderer
in search.ts (a rendering-ownership violation: a templ twin existed via
the HX-Request branch). The input now hx-gets layout=dropdown and swaps
the new templ SearchResults partial — titles/authors highlighted
server-side (escape-first, safe by construction), rows with the same
progress overlays — while a slim Alpine component owns only
client-local state (min-length gating, open/close, keyboard selection,
htmx lifecycle). web/src/rendering-allowlist.txt drops the search.ts
baseline to zero accordingly.
Integration test pins the per-user join (fresh user sees 1.0/0.42/null
on the fixture; the shared admin sees null everywhere — read-only) and
the dropdown partial's overlays/count/row count.
The /series page rendered its grid in one fixed order (newest first)
with no way to find a series beyond eyeballing cards. The page now has
a toolbar — series-name search, author filter, viewer-progress filter
and a 12-key sort select — wired as a plain GET form (the page's
existing model: the URL is the single source of truth, and the library
switcher now carries q/author/status/sort along on switch).
Extensibility is deliberate (user request): sorts live in a registry
(services.ResolveSeriesSort) mirrored by a CASE cascade in
GetDistinctSeries and a <select> in series.templ — a new sort is one
entry in each. The sort value is a BOUND PARAM compared inside the
cascade, never interpolated; unknown keys resolve to the default.
Filters: series-name matches in WHERE (the group key, so counts stay
true); author (BOOL_OR any-book match) and status (finished/reading/
unstarted vs the viewer's progress, same 0.999 threshold as the
filter_query status arms) in HAVING so COUNT(*) keeps reporting the
whole series. GetDistinctSeriesCount shares the JOIN + HAVING so
pagination totals match. The reading_progress join can't multiply rows
(the table is UNIQUE per media_item+user).
Cards gain the viewer's finished-count ("X of N read" + progress bar),
and the empty state distinguishes "no series at all" from "no matches —
clear filters". The JSON /api/series endpoint takes the same params and
gains read_count/last_read_at additively.
Integration test pins every sort key against a three-series fixture
with distinct created/author/count/progress answers, the filter matrix,
composition, unknown-sort fallback, card read-counts and the JSON
contract — all as the fresh regular user so the live reader's data is
untouched.
- TestRetryProcessingIssueRowless: a row-less import_failed entry retries
to a real import and resolves; a missing file retries to 409 with the
real reason and stays open; an item-backed issue retries through the
rescan mechanism and resolves.
- TestScanLibraryFolderPathValidation: an unregistered folder_path is a
400; a registered one scopes and enqueues (202).
- TestScanSingleFolderSkipsArchiveReconciliation: scanning one folder of
a two-folder library leaves the other folder's healthy item at
missing_scan_count 0 / unarchived — pinning the partial-scope guard
against the false-archive regression it exists to prevent.
Unidentified sync files used to be stuck: every KOReader re-sync
re-queues them, and there was no way to say "this isn't a book" or "I
don't want this one on the server".
Ignore is the durable dismiss: a new ignored_at column hides the row
from the page list, dashboard badge, and per-device API while the sync
keeps refreshing its metadata (the upsert never clears the column).
A collapsible "Ignored" section lists dismissed rows with Restore and
Delete so a misclick is recoverable. Delete is the hard exit — the row
is gone, and honestly re-queued if the device syncs that file again
(the confirm dialog says so).
Actions are page-scoped SSR routes: htmx swaps in the re-rendered
section so counts and buckets can never drift from the DB; ownership
is row → device → user (admins may act on any row). Integration tests
pin the lifecycle (hide everywhere, survive the sync upsert, restore,
delete), foreign-user 403s, and the render surface.
New users column surfaced as an On/Off dropdown in the profile form.
The form and API carry "true"/"false" strings with the theme-style
""-guard, so JSON clients that omit the field can't clobber it;
anything else is rejected. GET/PUT /api/auth/profile expose
keep_screen_awake alongside the other profile fields, and the reader
route threads the preference into its template metadata.
Integration tests pin the schema default, both values, omission
semantics, and rejection — dedicated test user, never testuser.
Repo-wide gofmt sweep: const-block alignment, doc-comment spacing,
statement-after-brace fix in media.go, trailing blank lines. No
semantic changes — whitespace-insensitive diff is empty for every
file in this commit.
Speed becomes a per-USER-per-BOOK preference that syncs across
devices, with a user-level default for books not yet played:
- new audio_playback_speed table (mirrors reading_speed's shape; a
preference, NOT a measurement and NOT a reading_progress column) +
users.default_audio_speed, schema in place
- chapters endpoint resolves per-book row → user default → 1.0 and
ships playback_speed + default_playback_speed (zero extra round
trips; default powers the dropdown marker)
- PUT /api/media-items/:id/audio-speed upserts (range-checked; the
client SPEEDS list stays the real gate; library visibility gates
writes)
- the user default rides the profile path like theme/timezone:
profile form gains a 'Default Playback Speed' select (server-side
SpeedOptions mirrors the web SPEEDS list), GET /profile carries it
- web: restorePosition applies the resolved server speed (localStorage
audio_speed demotes to a first-paint cache); setSpeed PUTs
fire-and-forget with a failure toast; the Speed dropdown gains a
'(default)' row marker and a 'Make N× my default' action with a
success toast
ALSO fixes a pre-existing profile-page bug surfaced by the §8 form
verification: the password section's current_password input was
required INSIDE the shared form, so HTML5 validation blocked every
Save Changes submission (dead since the shell migration). The
password flow keeps its own gates (disabled-button checkers + server
400).
Integration test TestAudioPlaybackSpeed pins the full contract (unset
→ 1x, upsert resolution, range rejection, default fallback for
row-less books, default updates re-satisfying unset books). Verified
live: 9 Playwright checks.
The testuser-experience rule: the suite must leave shared settings
exactly as it found them. setSystemSettingRestored snapshots the prior
system_settings row, writes through the sanctioned API path (the only
one that reloads the settings registry — raw SQL leaves the cache
stale), and registers a cleanup restoring what WAS there, never a
hardcoded value.
setExtractorVersion now rides it, so the ev-pin tests no longer
restore scan_extractor_version to a hardcoded 1: a deliberate dev bump
of the extractor version survives a suite run instead of being
silently cancelled (the live dev server obeys that setting on its next
poll and would stop re-extracting parser fixes). The now-redundant
explicit t.Cleanup restores are removed from
TestExtractorVersionReextract and pinExtractorVersionOne.
Three tests in scanner_audio_guard_test.go pin the routes the wipes
traveled; each drives a real scan/rescan/edit over ffmpeg-rendered
fixtures and asserts the payload survives with duration and manifest
intact:
- TestRescanMediaItemPreservesAudioMetadata: folder book through the
focused payload writer.
- TestRescanMediaItemPreservesAudioMetadataSingleFileM4B: the incident
class verbatim — single-file m4b (no layout key) routes through the
generic updater; fixture rendered with ffmpeg AAC/ipod
(writeTestTrackM4B).
- TestMetadataEditorSavePreservesAudioMetadata: the metadata editor's
full-row save through PUT /api/media-items/:id.
Verified to bite: with the handler fix stashed, the editor test fails
with "metadata edit must never NULL audio_metadata"; with it, green.
The extractor/force-rescan tests assert manifest stamps against the
default version (1), but the setting lives in the SHARED dev DB — any
prior run (or a manual admin-UI bump) can leave it at 2, failing every
stamp assertion through no fault of the code. Each ev-sensitive test
now pins the setting to 1 at start and restores 1 on cleanup; proven
against a deliberately bumped setting (tests pass, row restored).
Force rescan contract change (owner-directed): the editorial-keep skip
is retired. Force rescan now re-derives EVERY field on EVERY book from
all sources — multi-file books included (the folder/zip flows ran the
full chain on create and on manifest-flagged edits, but skipped the
sidecar apply under force). metadata_overrides is the only protection:
ApplyMetadataOverrides re-applies manual values inside the writer, so
manual edits survive; everything else re-derives.
- A force pass also STAMPS each re-derived book's manifest with the
current parser version (success-only — same truthfulness rule as
Phase 10) and closes its open processing issues (Phase 11 self-heal).
Force rescan and the ev-bump trickle now converge: either leaves the
library fully current; the deploy order is bump-first, then Rescan
All (documented in the plan and contract_pins).
- Single-file force rescan already ran the full chain — unchanged,
now pinned.
NEW filter field manually_edited ("Manually edited", boolean):
cardinality(mi.metadata_overrides) > 0 / = 0. Composed-engine (qf)
only — the builder's Book facts group gains it with is true/false;
no value menu, no suggestion surface; family lens inherited.
Tests: multifile force applies sidecar metadata + stamps + override
protection; single-file pin; the filter both directions scoped to the
test library; adjacent suites (Audiobook, Calibre, ExtractorVersion,
CrossLibraryMove, ProcessingIssues, FilterBuilder, SchemaIdempotency)
green. The extractor mechanism itself is untouched — the owner bumps
scan_extractor_version manually via the admin UI (plan Phase 13 §3).
processing_issues only recorded cross-library move rejections; files
that failed to import vanished into an error counter and stdout. The
ledger now records every failure class and heals itself:
Schema (in place, pre-release boundary):
- media_item_id NULLABLE + file_path TEXT (deliberately unbounded —
an over-long path is itself a failure mode; a VARCHAR(500) ledger
column died on the exact 22001 it existed to report — caught by the
new test). Row-less failures key on (library, file_path); item-backed
on (item, type). Widened dedup = COALESCE-expression unique index;
the old table UNIQUE stays (NULLs pairwise-distinct for row-less).
- Conflict-target upsert: repeat failures UPDATE the same row and
REOPEN resolved ones — never duplicates.
Wirings:
- CreateMediaItem failure (single-file + multifile createAudioBookItem)
→ row-less import_failed (error).
- Extraction hard-fail with fallback import → item-backed
metadata_extraction_failed (warning).
- Change-path re-extraction failure (the ev/sidecar guard) → item-backed
metadata_extraction_failed; LogProcessingIssue keeps its signature, a
row-less helper joins it.
Auto-resolution:
- Successful import → ResolveOpenProcessingIssuesForPath (library,
file_path).
- Successful (re)extraction → ResolveOpenProcessingIssuesForItem
(single-file change path, multifile editorial apply).
- Unchanged-skip scans do NOT resolve: extraction never re-ran, so the
degraded metadata is still degraded — the page heals exactly when the
book does.
Family lens on ALL issues surfaces (list, stats, resolve, delete, SSR
page): explicit out-of-set library id → 403, never silent. This
tightens one pre-lens behavior — a nonexistent library id returned
200-empty, now 403 (pinned, changelog entry in docs/contract_pins.md).
List is LEFT-JOINed so row-less rows appear (media_item_id "",
title falls back to the path).
Tests: full lifecycle state machine through real scans (fail → appears,
repeat → no dups, success-at-path → resolved, LEFT-JOIN list shape);
family lens 403s with a fresh non-shared admin; all 7 pre-existing
issue suites green (two expectations widened to the lens, changelogged);
adjacent suites (CrossLibraryMove, Audiobook, Calibre,
ExtractorVersion, SchemaIdempotency) green; unit suites green.
setupTestServer built the settings registry (and wired it into
systemSettingsHandler) but never called authHandler.SetSettings, unlike
main.go:112. The auth handler therefore ran with a nil registry and
defaultTimezone() took its nil-fallback branch, returning "UTC" for every
Register/AdminCreateUser in tests regardless of the system default_timezone
setting — so the 740213f0 inheritance pin in TestAdminCreateUser could never
pass and failed every full-suite run.
Also replace the TestAdminCreateUser "stable-ish" uniquifier (MaxConns is
pinned to 1 by the harness, so it produced the same user every run and
depended on the next run's cleanup sweep) with a real uuid suffix.
Parser fixes (7e4c97f6, genre model v2) never reached rows ingested
under an older parser: the sidecar never changed, so every fingerprint
said 'unchanged'. The manifest JSONB now carries the reserved 'ev'
stamp — the parser version that produced the item's row.
Semantics (all pinned by tests):
- ABSENT ev decodes as 1 ('v1-or-unknown'); stamping is truthful-only:
create paths + post-extraction stores stamp the setting's current
value; deploy-day backfill and the extract-failure guard stay UNSET
(a row never claims a version its data can't prove — a mid-backfill
bump self-heals next poll); repoint reconciliation preserves the
stored ev (stat-only refresh re-extracted nothing).
- NEW setting scan_extractor_version (seeded + registry + getter,
int default 1, category scanner): a manifest whose ev != the setting
re-extracts through the SAME change path (full extraction →
updateMediaItemWithMetadata → fresh stamped manifest). Sidecar-less
books get an entries-less manifest as their ev home (create: stamped;
first-sighting: unset).
- Catch-up bounded: evReextractBudgetPerScan=100, atomic (watcher pass
can overlap a poll scan), reset per ScanFolders — amortized +
resumable across polls; no force rescan, no flood.
- Multifile books included: the ev gate rides multifileSidecarChanged's
shared ladder; ev-driven manifestChanged drives the editorial apply
+ stamped re-store.
Settings API: the unified GET/PUT /api/system/settings now lists the
key (registry-driven, auto-surfaces in the admin UI). Test harness:
the registry is now Loaded + wired into the SystemSettingsHandler
(main.go parity) — settings writes through the API were 503s before.
Verified: unit (ev normalization, budget CAS, GetExtractorVersion
cache paths, registry clamping); integration state machine —
create-stamp ev=1 → equality skips (corruption probe persists) →
wiped scan-state backfills UNSET without re-extract → bump re-extracts
ONCE (sidecar value restored, ev=2) → stable (second corruption
persists); multifile twin green; adjacent suites (Audiobook, Calibre,
CrossLibraryMove, SchemaIdempotency) green; live app restart re-scans
1198 files with 0 changed (deploy-day rule), setting listed at
/api/system/settings.
The browse-on-click blank sentinel was trimmed to empty (correct), but
the typed query had already been bound unconditionally — the id-source
suggestion query has no score CASE to reference it, leaving an unused
parameter Postgres rejects (42P18). Every click of an empty library /
collection / added-by box answered 'No matching values' via the error
path.
- ComposedFieldValues binds the typed query ONLY in branches that
reference it and only when non-blank; the score expression is
'0 AS score' for browse (no param) and the similarity CASE when
typing. composedIDSourceValues takes the raw typed string and binds
it only when the name filter is built. userBind already followed
this discipline; audited every bind site in the file (fuzzy helpers,
composed search, count) — all interpolate immediately.
- Tests: library browse under rules (exactly the surviving library,
count pinned), series browse blank, added_by browse blank (no
error), collection browse blank (offered with count) and under a
zero-match rule (never offered). Verified end-to-end against the
live dev data: author+Sapkowski AND narrator+Kenny → Audiobooks (8).
- Blank typed queries rode the STATIC autocomplete path for the classic
six fields, where a single space similarity-matches into nothing —
clicking an empty value box with no other filters said 'No matching
values' in a library full of values. Blank = browse now always rides
the composed engine (which trims and answers the top values); static
keeps its pinned contract for real typed queries. Regression-pinned:
browse with no rules lists the field's values.
- The value popup now drops BELOW the input (top: 100%), not over it.
- Field-menu group headers lost position:sticky — the search box alone
stays pinned.
- Per-rule connectives: the chip before each rule flips that rule
between AND and OR — OR-joined neighbors merge into an invisible
OR-block, AND pulls them back out. The tree stays the single source
of truth (implicit blocks are a client marker the server ignores),
so deep links, saved filters, and the composed engine are untouched.
Group-mode pills are gone; explicit groups remain as pure structure
(+ group). Compile limit, same as Excel: flat chips express
AND-of-OR-groups; a top-level OR of AND-blocks still needs nesting.
- Browse-on-click: clicking an empty value box now fetches the field's
top values UNDER the other rules (blank typed = browse, not a
similarity filter) — 'assign an author, click Publisher, see their
publishers' without typing. Server: the composed suggestion engine
trims the typed query; client: the >=2-char guard no longer blocks
the empty query, browse limit 15 vs 8 while typing.
- Tests: the browse case pinned (blank sentinel + rules → the field's
values under those rules, zero-match values absent).
- Field registry (services/filter_registry.go): ~55 typed fields —
every book column, file/lifecycle facts, the three HOLDING
STRUCTURES (library, collections with excluded-aware membership,
series with FULL primary+alternate_info membership), and the whole
user-relative class (status, progress, last read, my rating, review,
notes/highlights presence, note/highlight text, time spent, pages
read, collections, added by) reading the VIEWER's rows. Operators
are typed constants; negation is an operator.
- Composed engines (services/filter_query.go): one Go clause builder
over raw pgx feeds the search (identical SearchMediaItemsUnifiedRow
shape), the field-aware suggestions, and a true-COUNT live count.
Mandatory WHERE head everywhere (family lens + integrity facts).
Values are always bound parameters — no user string reaches a
field/operator position, ever.
- Wire: the qf JSON param (validated strictly — unknown
fields/operators/malformed values are 400); qf wins over flat params
on hand-crafted URLs; flat deep links keep working unchanged.
- Endpoints: GET /api/media-items/filter-count (live count);
field-values autocomplete upgraded in place — any registry field
key, qf binds the OTHER rules (symmetric under-filter suggestions,
zero-match values never offered); classic six without a tree keep
their static pinned contract.
- UI: the drawer becomes the rule builder — the ONLY filter UI (templ
markup, Alpine tree state, sanctioned island; grouped field menus,
operator menus with red negations, field-aware value menus, live
count, chips per rule with red negations, URL rehydration, smart
serialization flat-when-simple else qf).
- Tests: composed semantics (AND/OR/negation/nesting/enums/bools/
numbers/dates/empty), every user-relative field class with the
my-data weld (two readers, different worlds), note-text substring
pin (catharsis ≠ Carthage), family lens (hidden libraries nowhere,
403s), suggestion-under-rules exactness, qf validation + precedence,
flat-vs-tree equivalence; registry completeness unit test.
- Remove the Phase 7 facet-panel UI: SSR route, facet_panel.templ,
FacetCountsForView, drawer panel block, panel TS + .facet-* CSS.
The ENGINE stays (FacetCountsUnderFilters, all-facets endpoint,
drift guard) — it becomes the Phase 9 builder's machinery.
- Fix the All-Books bug: field-values autocomplete (author/genre/
tags/narrator/series/language) treats library_id as OPTIONAL —
absent aggregates every VISIBLE library (family lens in-query);
explicit out-of-set library_id is now 403, never a silent pass.
The four scalar Search*Values queries get the nullable shape.
- toast.ts: suggestion fetches are exempt from error toasts
(console-only) — a failed background hint never interrupts.
- TestFacetPanelPartial deleted with the behavior; TestFieldValuesAll-
Libraries pins the fix (200 + cross-library aggregation, family
lens, 403). Contract-pins changelog entry added.
The 9 Batch A facets could not be SET from the web UI, the facets
endpoint had no in-app consumer, and the registry's libraryTypes hints
rendered nowhere. This phase closes all three.
- ONE UNION-ALL sqlc query (FacetCountsUnderFilters) computes all 14
listable facets in a single round trip: each arm omits its OWN
facet's filter block and carries every other registered facet's
block (copied mechanically from SearchMediaItemsUnified) — classic
faceted semantics: with filters active each facet counts under the
others; with none, blocks are neutral and the counts are the v1
global-per-library ones. One query serves both cases. The 14
per-column Batch C queries are deleted.
- Endpoint: no facet param → the new all-facets shape
({facets: {param: [{value, display?, count}]}}), filters bound via
the shared ParseMediaSearchParams, library_id optional (the
All-Libraries view, family lens still enforced); facet=X keeps the
pinned single-facet shape (now routed through the same query with
neutral filters).
- SSR partial GET /media-items/facet-panel: templ renders the drawer's
value lists — registry order, Batch D library-type scoping
(FacetsForLibraryType: manga_type on manga, reading_direction on
manga+comics, narrator on audiobooks; All view = union), prettified
genre labels + language display names, active values marked. Family
lens: out-of-set explicit library_id is 403.
- Drawer wiring: the panel fetches on drawer open and after every
apply with the CURRENT filter query string; a value click toggles
that facet's comma-joined filter (hidden input where the drawer has
no field — same as deep links) and re-applies the form; chips and
URL stay in sync. Values are server-rendered (rendering doctrine);
bookshelf.ts owns only the click/fetch orchestration.
- prettifyGenre moves to internal/utils as the canonical
implementation (services' display enrichment shares it; templ wraps
it; the TS port in bookshelf.ts is unchanged).
- Drift guard extended: FacetCountsUnderFiltersParams must carry every
registry facet's param, the bools as pgtype.Bool.
Tests: under-filter exactness (own-block omission, OR-within,
AND-across), family lens (hidden values absent; 403 on both shapes),
all-shape + pinned-shape coexistence, partial rendering with Batch D
scoping + active marking + 403. Full container suite green except the
documented pre-existing TestAdminCreateUser red.
Two suite failures, neither caused by scanner code (the Phase 6 diff
touches no handler):
- search_test invalid_library_id expected the pre-family-lens 404;
7e586df4 shipped the AGENTS-mandated 403 for every out-of-set id
(existence must not be enumerable) and docs/contract_pins.md:195
documented it — this one subtest's expectation was missed. Pin
widened to the shipped 403.
- TestWebSocketPingPong deleted: it asserted a text-level ping→pong
exchange no server code has ever implemented and no client has ever
sent; it passed vacuously through its read-timeout path until
interleaved scan_progress broadcasts from the live server's scans
reached the socket first. Keepalive is the standard protocol-level
ping the hub already sends. Never-built behavior is not a pinnable
contract.
Both recorded in docs/contract_pins.md. Full container suite is green
except the documented pre-existing TestAdminCreateUser red.
Reverses the Phase 4 scope cut: for EXISTING folder/zip audiobooks a
sidecar edit was invisible to everything, including force rescan —
updateAudioBookItem writes only scan-derived fields, and these formats
often have no useful embedded tags, so the sidecar is the only source.
- Change fingerprint gains the manifest compare in both processors:
changed := size differs || track payload differs || manifest drifted.
The ladder runs against the bulk-loaded map entry (stat pairs only,
zero extra queries); NULL stored manifest backfills without
re-extraction (deploy-day rule, same as single-file).
- Manifest-changed applies the once-extracted book metadata to the
editorial columns (updateMediaItemWithMetadata, overrides protected)
and re-runs the playlist hook. The apply runs BEFORE the focused
payload write: the generic update's format-group pass derives mime
from the sentinel (none) and chapters from metadata.ChapterCount
(unset) — the payload write must win (rescanMultiFileAudioBook's
ordering). Track-only changes keep the existing editorial-keep
update; force rescan keeps it too.
- Manifests seeded at multifile create (folder names derive from the
sentinel — the exact representative the extractor reads — via
detectSidecarChangeWithRepresentative, so manifest and extractor can
never disagree; a directory-basename cover is ignored by the
extractor and therefore not tracked).
- Zip parity fixes: the fast path now lifts archive state like the
folder path always did (a returned zip book stayed archived
forever), and the change branch handles the unchanged-archived case.
- Single-file sidecar change path: extract-failure guard — the old
non-nil empty &MediaMetadata{} fallback bypassed the update path's
filename-title guard (preExtracted == nil only) and would blank the
title on a failed extraction; the row is now left untouched and the
manifest refreshed. Empty-success extraction (sidecar removed from a
tag-less book) now passes nil so the create-equivalent chain (tags →
filename) re-derives instead of blanking.
- Multifile changes now count in the scan summary (changedItems);
they were previously uncounted.
Tests: sentinel-name derivation unit pin; folder sidecar edit
(title + collection sync + fresh manifest + steady-state skip),
track-only change keeps editorial, zip sidecar edit + archive
restore, sidecar removal falls back to filename (the wipe-guard's
testable sibling). Full container suite: green except the documented
pre-existing TestAdminCreateUser red.
Part 2 of the faceted plan, Phase 4 (steps 3-4). Size-only detection
was blind to metadata-sidecar edits; a size-stable book is changed by
EDITING its metadata.opf/metadata.json/cover — previously invisible
until a force rescan.
- Sidecar manifest fingerprints (media_scanner_manifest.go): the
manifest stats exactly the names the extractors consume (metadata.opf,
metadata.json, findSidecarCover patterns incl. <basename>.<ext>), so
manifest and extractors can never disagree. Shape v1:
{v, entries:[{n, s, m (unix nanos — second granularity would hide
edits landing in the backfill scan's own second), h (sha256 — covers
included: the repoint ladder needs a baseline hash for any stat-
drifted sidecar; a hashless cover turns harmless mtime drift into a
re-extraction, the exact flood this design kills)]}.
- Evaluation ladder in cost order: stat pairs identical → skip with
ZERO bytes read; stat drift → hash that sidecar (equal = host repoint
reconciliation, stat pair refreshed, no re-extraction; differing =
changed); NULL stored manifest → backfill only, NO re-extraction
(deploy-day rule: the first scan after release must not look like an
mtime flood).
- Change path: re-read full row, extractMetadata once and pass it
through updateMediaItemWithMetadata (preExtracted — metadata_overrides
stay protected), run syncPlaylistCollectionsAfterCreate (Correction 2:
a reading-order entry added to an EXISTING book now creates its
collection; add-if-missing is idempotent), store a fresh full
manifest. Creates seed their manifest at insert time.
- Multifile folder/zip books intentionally keep Phase 3 behavior: their
change path does not consume sidecar extractors, so gating them would
flip books 'changed' without picking anything up (plan correction —
the plan's §3 multifile note assumed otherwise).
- UpsertMediaScanState + GetMediaScanState queries.
Tests: media_scanner_manifest_test.go (ladder unit tests — fast path,
repoint, same-size drift, appear/vanish, unhashable legacy entries);
TestScanManifestLifecycle (seeding, unchanged-skip stability, repoint
reconciliation with stat-pair refresh assertion, same-size edit
consumed, playlist sync on the change path + idempotency, no-leak
guard). Container runner.
Part 2 of the faceted plan, Phase 3 (steps 1-2). The periodic scan
issued one GetMediaItemByFilePath per scannable file (~3,094 queries
per scan in prod, nearly all answering 'unchanged, skip') plus one
GetLibraryByFolder per file for folder resolution.
- NEW media_scan_state table: the scanner's per-item fingerprint
cache. Deliberately NOT a media_items column — the library-scoped
list emits raw sqlc rows as JSON (7537063c), so scanner bookkeeping
must never leak into API payloads nor churn the model or pins.
Manifest JSONB shape v1 documented in the schema (n/s/m/h; ev
reserved for the Phase 4+ extractor-version mechanism).
- GetScanStateByLibrary: lean bulk projection (id, file_path,
file_size, archived_at, missing_scan_count, file_sha256, manifest).
archived/hash columns ride along so the skip path's archive-restore
and hash self-heal checks stay DB-free too (refines the plan's
4-column sketch; same goal, narrower invariant).
- ScanFolders: per-run reset, per-folder library cache, one bulk load
per distinct library. processMediaFile: map hit answers existence +
size + flags with zero DB reads; force-rescan and size-change
branches re-read the full row (the change path may be expensive);
write-through on create/size-change keeps later passes honest.
- Multifile folder/zip books keep their full-row lookup: it is one
query per BOOK ROOT (not per track) and needs audio_metadata for
the duration-probe check. They join the map in Phase 4, when the
manifest compare replaces their fingerprint check.
- nil map (watcher's scanDirectory path, bulk-load failure) falls
back to per-file lookups — behavior-preserving; the failure case
logged once per scan.
Test: TestScanStateBulkLoad (unchanged rescan leaves updated_at
stable; API-created rows are seen by the bulk load; size changes
still update in place). Container runner.
Phase 1 + backend of Phase 2 of the faceted-navigation plan. One
declarative facet registry (services/facet_registry.go) drives the
bookshelf filter system; a drift-guard unit test reflects over the
generated SearchMediaItemsUnifiedParams so a registry entry and its
sqlc.narg block can never disagree silently.
- Nine new filter params on SearchMediaItemsUnified: story_arc,
imprint, edition (scalar-fuzzy), age_rating, format_group,
manga_type, reading_direction (exact), explicit, abridged
(presence-bound bools). Comma = OR-within multi-select on every
text facet; AND-across-facets unchanged.
- Genre model v2 (user rule: a subgenre is not a genre, it's a tag):
genre_lexicon.go rebuilt as shelf set + subgenre->shelf map;
pickSmartGenre picks direct shelf > mapped subgenre > Fiction floor
> empty (unknown-only stays tags-only). Type-words (Manga, Comics,
Graphic Novel) join the demote list. Both 'adventure' and 'action &
adventure' are shelves — provider data uses both and the verified
traces pin them.
- Language filter UX: comma elements normalize through the ingest
language map, so display names ('English') match rows stored as ISO
codes ('en'). The language autocomplete now serves display names.
- NEW GET /api/media-items/facets?facet=X&library_id=... — registry-
driven value+count listing (14 static per-column queries, family-lens
join); language values carry display names; bool/range facets 400.
- Family-lens gates: an explicit hidden-library library_id is now a
403 on the search endpoints (was a silent empty set) — JSON branch
and HTMX grid partial both.
- handlers.ParseMediaSearchParams is the SINGLE param parser for the
search JSON endpoint and the HTMX grid partial; the HTML branch
previously dropped publisher_filter and narrator_filter silently.
Pins: docs/contract_pins.md changelog entries (additive request
params; new facets endpoint). Tests: facet_registry_test.go (shape +
SQL drift guard + library-type union), series_parsing_test.go genre-v2
cases, facets_integration_test.go (param filtering e2e, counts,
language display, 403s) — container runner.
Completes the A2c acceptance scenario: click an author on a book
detail page -> land on the bookshelf filtered to everything that
author wrote (including their share of co-authored books).
- Media detail + unscoped list resolved maps expose subtitle,
explicit, authors, publishers (the library-scoped raw branch and
the search endpoint already did — raw rows follow the SELECT).
- SearchMediaItemsUnified SELECT carries the new columns; the search
endpoint serializes the row directly, so clients get them there too.
- book_detail.templ renders each author as a link to
/bookshelf?author_filter=<name>.
- Bookshelf honors filter params in the URL on load (author_filter,
series_filter, genre_filter, publisher_filter, tags_filter,
narrator_filter, language_filter): pre-fills the form, applies it to
the first grid load. Comma-joined values are multi-select via the
server-side string_to_array split. Filtered views are shareable.
Pins widened: search rows (+7 keys), resolved keys (+4); changelog
entries in docs/contract_pins.md. Android follow-up plan written to
~/.opencode/plan/bookhoard-app-v2-compat.md.
Phase 2b + 3 of the metadata overhaul (plan §2.4, §3.1-3.7).
Auto-playlist collections (§2.4): the scanner turns obviously-playlist
sidecar series (explicit '(Reading Order)'/chronological/publication/
in-order/timeline markers only — precision over recall) into curated
static collections: upsert by view_settings.imported_from_series then
name (rename-safe), position set ONLY on insert (clean integers; 1.5/1a
stay unpositioned rather than guessing), membership add-if-missing,
never removed or re-ordered by rescans, provenance in description +
view_settings, show_on_dashboard true (dashboard-eligible by design),
'scanner.auto_playlist_collections' setting (default on, Settings
Registry + schema seed).
Ordering (§3.1-3.2): collection listings order position ASC NULLS LAST
then added_at ASC (was added_at DESC — walked playlists backwards);
PUT /api/collections/:id/items/order applies full-order rewrites
(owner-only, service-gated).
hide_finished + Up Next (§3.3-3.4): GetCollectionItemsForDashboard
gains viewer_user_id + hide_finished (LEFT JOIN reading_progress,
COALESCE percentage < 0.999) and viewer_percentage rides the row for
card graying; GetCollectionUpNext serves the collection's reading-queue
head with the family-lens visible-library array enforced.
Dashboard (§3.5): user collections are dashboard-ELIGIBLE BY DEFAULT —
the show_on_dashboard gate (never writable for user rows since the
column shipped) is gone from GetUserCollectionsForDashboard; hiding
stays dashboard-prefs (hidden_collections) territory; hide_finished
filtered-everything now emits a Complete section instead of silently
vanishing; sections JSON carries is_playlist/complete.
Series page ordering hardening (§3.6): GetSeriesBooks tiebreaks by
title ASC — NULL/tied numbers previously returned heap (scan) order,
which is what the user saw.
Frontend custom-section-builder note: the builder has sent
show_on_dashboard: true since its creation; the API never bound it.
User-row visibility is now intentionally prefs-governed, so the field
stays unbound by design (documented in code).
Pins widened (docs/contract_pins.md): collection detail + up_next;
dashboard sections + is_playlist/complete; new reorder endpoint
recorded.
Verified: TestPlaylistCollectionsEndToEnd (new, full e2e: sidecar
series split, collection auto-creation with position 7, rescan
idempotency, owner-only reorder, finished-filter + completion marker +
Up Next drain) plus the full affected integration sweep green.
John O'Keefe