96fd2aad424e94996940d89bab6f5c7493c673e3
1973
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 John O'Keefe
|
96fd2aad42 |
feat(audio-player): scrub interaction rework — free-flow drag, pixel magnetism, touch pull-away precision (plan §9)
Mouse: drag is free-flow 1:1 (rail only on Shift+drag, now with window from/to edge labels); clicks land with pixel magnetism (MAGNET_RADIUS_PX=10) — the old 5-second threshold was sub-pixel on long books and never fired, which was the 'chapter scrub isn't accurate' bug. Hover inside the radius locks the tooltip to the line with a highlight. Shift+click jumps to the nearest line from anywhere. Touch: drag is an incremental cursor (applyDrag) whose scale tightens continuously as the finger pulls off the bar (dragScale, exponential 10^(-|dy|/80px)) — the rail is the feedback: window shrinks with the zoom (touchRailSpan) and lifts toward the finger (railLiftPx). Dialog retitled 'Shortcuts & interactions' with a Mouse & touch section driven by POINTER_GUIDE (audio-keys.ts). Old seconds-threshold snapToChapter deleted. 133 unit tests; 19 Playwright checks on the KJV (1189ch) and Gardens (34ch) books. |
||
|
John O'Keefe
|
1f30602c23 |
feat(audio-player): progress bar overhaul — ticks, buffer, wheel, tooltip, precision rail (plan §1)
The scrubber becomes a layered bar: the native <input type=range> stays on top for a11y (focus, arrow keys, aria — its track goes transparent via static page CSS) while the layers beneath render the state: - Chapter ticks: density-decimated (denseTicks keeps only marks >= 4px apart) so an 1189-chapter 86-hour book gets ~100 readable marks instead of a solid mass; the current chapter's tick is tinted. - Buffered span: the ACTIVE track's buffered-ahead end mapped into the global bar (buffers are per-track and never span the whole bar). bufferedAheadEnd reads the range containing the playhead — a huge m4b's moov index at the file tail is its own buffered range and must not read as 'the whole book is buffered'. Rides progress + the events that demonstrably fire for Range-served files that go network-idle while paused (timeupdate/canplay/loadedmetadata/playing). - Wheel over the bar: ±5s nudge, Shift = ±60s. - Hover tooltip: time + chapter title under the pointer. - Precision rail: while dragging, a ±5-minute window floats below the bar (fixed anchor; release and re-grab to travel far) and the drag maps at minutes-per-pixel instead of hours-per-pixel — the whole point for 80h+ books. Click snaps to a chapter line within ~5s (snapToChapter). Pointer handling is ours end-to-end: pointerdown maps hours-per-pixel + snaps, captured moves remap through the rail, release seeks. The native drag default is prevented so the two mappings never fight; keyboard arrows keep their native step and seek via change. Time labels get reserved width — the bar no longer resizes when the elapsed label grows from 0:00 to 43:07 mid-drag (caught by the rail drag check measuring a 25% mapping drift). Pure math in audio-scrubber.ts: pointer mapping, chapter lookup, snap, tick density, per-track buffer→global span, rail window/mapping/ticks. Unit: 16 new tests including both book shapes (86h/1189ch and 26h/34ch). Live: 23 Playwright checks on KJV (folder) and Gardens of the Moon (single-file m4b); zip books share the folder manifest shape (no zip fixture exists in the dev library). Probe user deleted. |
||
|
John O'Keefe
|
18bed79d3a |
feat(audio-player): settings popover + autoplay-on-open (plan §4)
Header gear (beside the keyboard icon) opens a 'Player settings'
popover — one option row today, structured for more: 'Autoplay on
open', a role=switch toggle persisted to localStorage audio_autoplay
('1'/'0', default off) via the normalizeAutoplay one-source helper.
maybeAutoplay() runs strictly AFTER restorePosition resolves: the
?loc=audio:<seconds> deep link keeps winning on position; the setting
only decides whether playback starts. A browser autoplay-policy
rejection (or any play() failure) is swallowed silently — the user
just taps play; the unhandled-rejection trap in init would otherwise
shout 'REJ:' into the title. settingsOpen joins closeTopSurface, so
Esc closes the panel with the other surfaces.
Unit: normalizeAutoplay tests. Live: 13 Playwright checks — default
off stays paused, toggle + persistence, reload autoplays with no
gesture, deep link lands at 600s global while autoplay starts, off
stays paused, and a stubbed NotAllowedError proves the silent
swallow; probe user created and deleted.
|
||
|
John O'Keefe
|
fe7eab91e8 |
feat(audio-player): sleep timer — moon pill, countdown, 5s fade then pause (plan §3)
Off / 5 / 10 / 15 / 30 / 45 / 60 minutes / end-of-chapter. The moon pill sits in the settings strip between Bookmark and Volume and shows the live mm:ss countdown while armed (or 'End of chapter'); the dropdown's armed row highlights like Skip/Speed. audio-sleep.ts is the pure, DOM-free state machine (arm/due/fired/ fade-ramp/label + the SLEEP_CHOICES vocabulary the templ dropdown iterates). The component owns the timers: a 1s countdown ticker in minutes mode, and a ~5s element-volume fade on expiry that ends in audio.pause() — the SAME pause listener a manual pause fires, so the progress push and media-session state land. The user's configured volume is restored right after the pause, and the fade operates on the element's volume the user controls, not a parallel path. End-of-chapter fires when the armed chapter is left — natural crossing (tickChapter) or a manual jump (jumpToChapter) alike. Manual play during a fade aborts the expiry (you're awake, you asked); stopSleep restores any partial fade. Per-session ONLY: nothing touches localStorage — the reload check proves it starts Off. sleepOpen joins closeTopSurface's dropdown branch, so Esc closes it with the other settings-strip surfaces. Muted (or zero-volume) expiry skips the pantomime and pauses immediately. Unit: audio-sleep.test.ts (deadline mechanics, chapter-fired semantics both directions, fade ramp clamps, label formatting, SLEEP_CHOICES integrity). Live: 24 Playwright checks on the KJV book — dropdown, countdown ticking, armed-row highlight, end-of-chapter fire + reset, internal 3s deadline fade→pause→restore, play-abort, reload-starts-Off; probe user created and deleted. |
||
|
John O'Keefe
|
295c4b0664 |
feat(audio-player): keyboard shortcuts — declarative keymap, dialog, media keys (plan §2)
Space/Enter (body-focus only), H/L + arrows skip, J/K + PgDn/PgUp chapter steps, B bookmark capture (note box auto-focuses, Enter saves even when blank, Shift+Enter inserts a newline, Esc cancels), Alt+arrows / Alt+H/L speed steps, up/down volume with a brief OSD, M mute, [ ] skip-duration cycle, ? shortcut dialog, Esc closes the top-most surface (help → capture → note edit → drawers → dropdowns). audio-keys.ts is the ONE declarative table: resolveShortcut() drives the document keydown dispatcher and visibleKeymap() renders the templ dialog (x-for over the exposed keymap — templ cannot import TS). Hardware media keys ride the existing MediaSession handlers; the Media* keydown codes resolve through the same table to the SAME methods, so both layers stay one behavior. Guards mirror the ebook reader: typing in any INPUT/TEXTAREA/contentEditable never triggers shortcuts (bookmark notes included), a focused button keeps Space/Enter for itself, and the focused scrubber keeps its arrows (its own @change seeks — a native arrow nudge moves 0.08s, not a 10s skip). SKIP_SECONDS consolidates to one const beside SPEEDS (the templ skip dropdown now iterates skipChoices; [ ] cycles via cycleSkip()) — the same one-source pattern as the §5 speed-list consolidation. Unit: audio-keys.test.ts (resolution, Alt disambiguation, media keys, keymap integrity and conflict-freedom, dialog visibility) and cycleSkip tests. Live: 39 Playwright checks on the KJV folder book covering every binding, the guards, dialog and OSD; probe user created and deleted. |
||
|
John O'Keefe
|
db96ea91cd |
refactor(audio-player): one source for the speed list (plan §5)
The rate list was duplicated — hardcoded in the templ x-for AND implicitly in audio.ts — and phase 2's keyboard stepping would have needed a third copy. SPEEDS is now the single exported const (the agreed nine values, 0.5 through 3); the component exposes it as 'speeds' for the dropdown's x-for (templ cannot import TS), and stepSpeed() walks the same array for the future audio-keys consumer. normalizeSpeed() maps persisted values onto the list: garbage or out-of-list localStorage entries fall back to 1x, and init() writes the normalized value back so storage heals instead of staying poisoned. Pure logic unit-tested in audio.test.ts; live-verified with Playwright on the KJV folder book: nine options render, pick/reload/persist and the stale-value fallback all behave, button label and audio.playbackRate agree in every case. |
||
|
John O'Keefe
|
408a972665 |
fix(audio-player): playback speed silently reverted to 1x on every reload
restorePosition() swaps the <audio> element's src (manifest track for multi-file books, whole file for single-file, plain URL on offline start), and loading new media resets playbackRate to the element default. init() had applied the persisted speed before any src existed, so after the swap every load actually played at 1x while the Speed pill still showed the stored value — pick 1.5x, reload, and audio was back at 1x. The seekTo() src-swap path already re-applied the rate; the initial load paths now do the same (manifest branch + offline-catch branch). Surfaced by the phase-1 speed-list browser check: button label and audio.playbackRate disagreed after reload. |
||
|
John O'Keefe
|
6508abc00f |
docs(agents): the doctrine — tests never run against a real server
Codify the standing rule at the top of the Tests section: the dev DB is the test environment, and the invariant every test must uphold is the testuser experience — its settings, scans, and reading data must be exactly as the suite found them. The helpers exist to enforce this; when a test needs protection the helpers don't provide, expand the helpers rather than hand-rolling. New known-gotchas from the 2026-10-21 incident cluster: 7. scanner re-extraction is write-after-success — audio_metadata is guarded on both writers; new scan-owned column writes need the same guard. 8. templ double-escapes style attribute values (sanitizer + EscapeString) — emit quote-free CSS via unquoted url() tokens through SafeCSS. 9. vp test runs the vite config — build-hook plugins fire in the test context too; outDir-mutating plugins need an "am I a build?" guard. |
||
|
John O'Keefe
|
3ed852515b |
test(helpers): setSystemSettingRestored — leave shared settings as found
The testuser-experience rule: the suite must leave shared settings exactly as it found them. setSystemSettingRestored snapshots the prior system_settings row, writes through the sanctioned API path (the only one that reloads the settings registry — raw SQL leaves the cache stale), and registers a cleanup restoring what WAS there, never a hardcoded value. setExtractorVersion now rides it, so the ev-pin tests no longer restore scan_extractor_version to a hardcoded 1: a deliberate dev bump of the extractor version survives a suite run instead of being silently cancelled (the live dev server obeys that setting on its next poll and would stop re-extracting parser fixes). The now-redundant explicit t.Cleanup restores are removed from TestExtractorVersionReextract and pinExtractorVersionOne. |
||
|
John O'Keefe
|
f62b46fb15 |
test(integration): route regression tests for the audio_metadata guards
Three tests in scanner_audio_guard_test.go pin the routes the wipes traveled; each drives a real scan/rescan/edit over ffmpeg-rendered fixtures and asserts the payload survives with duration and manifest intact: - TestRescanMediaItemPreservesAudioMetadata: folder book through the focused payload writer. - TestRescanMediaItemPreservesAudioMetadataSingleFileM4B: the incident class verbatim — single-file m4b (no layout key) routes through the generic updater; fixture rendered with ffmpeg AAC/ipod (writeTestTrackM4B). - TestMetadataEditorSavePreservesAudioMetadata: the metadata editor's full-row save through PUT /api/media-items/:id. Verified to bite: with the handler fix stashed, the editor test fails with "metadata edit must never NULL audio_metadata"; with it, green. |
||
|
John O'Keefe
|
35d8ba32e5 |
fix(media): metadata editor save no longer wipes audio_metadata
PUT /api/media-items/:id (the metadata editor) writes the FULL row via UpdateMediaItem, whose query sets audio_metadata unconditionally — and audio_metadata has no editor field, so the params literal left it nil and every save NULLed the payload. A plain metadata edit (genre tweak, title rename) dropped any audiobook back to "1 second long" with no track manifest: the same incident class as the scanner wipe, sitting on the most-used admin path in the app. Carry existing.AudioMetadata through the write. Write-after-success: only the audio chain (probe-backed extraction / UpdateMediaItemAudioBookPayload) ever replaces that column. |
||
|
John O'Keefe
|
80bfc1a77b |
fix(scanner): re-extraction is write-after-success — no NULL audio_metadata
The 2026-10-21 wipe: a re-extraction pass NULLed audio_metadata on the m4b audiobooks — every affected book dropped its chapters, track manifest and duration and displayed as 1 second long (the reader also lost its manifest, since single-file books synthesize it from the payload). Guards on both writers of that column: - updateMediaItemWithMetadata (generic full-row pass): when the new extraction carries no audio payload, keep existing.AudioMetadata. Single-file m4b books route here, and a failed/absent extraction leaves AudioMeta nil. - updateAudioBookItem (focused payload write): an empty payload (no tracks, no chapters, no duration) over a populated row is a failed extraction — refuse the write with a loud warning and let the next scan retry. audioPayloadIsEmpty() pins the empty-payload predicate (unit-tested): the column is only ever replaced by fresh data from a real audio extraction, never by absence of one. |
||
|
John O'Keefe
|
970beee4b8 |
fix(pwa): add the manifest icons the manifest always referenced
manifest.json has referenced /static/icons/icon-{192,512}.png since
April, but the directory was never committed — ~100 404s per browsing
session and a broken PWA install. Rendered both sizes from the
existing favicon book glyph on the Tokyo Night palette
(#1a1b26 background, #7aa2f7 stroke).
|
||
|
John O'Keefe
|
fa5b952c54 |
fix(audio-reader): valid doctype — page loaded in quirks mode
audio_reader.templ was the only template emitting <!DOCTYPE html/> (trailing slash is a parse error), which put the audio player page in quirks mode — document.compatMode was BackCompat while every other page is CSS1Compat. Quirks layout rules are a standing hazard for the player's h-screen/flex page even where it happens to render. Drop the slash. |
||
|
John O'Keefe
|
ff17d14694 |
fix(book-detail): hero backdrop — templ double-escapes style values
templ's codegen wraps style attribute expressions in a sanitizer AND
templ.EscapeString, so any quote character — even inside templ.SafeCSS
— reached the DOM as literal ' text. The hero backdrop's url('...')
never parsed: no blurred cover on any book detail page, and Firefox
resolved the mangled token as a relative URL, fetching /media/& (500)
once per page load.
Fix: emit quote-free CSS — an unquoted url(...) token built through a
new cssURL() helper that percent-encodes the characters that would
break it (space, quotes, parens, backslash). Stored cover paths are
already percent-encoded URL paths, so the escaping is defensive only.
|
||
|
John O'Keefe
|
807899b137 |
fix(reader): arrows stay page turns while the progress slider holds focus
Clicking the bottom progress slider left it focused, and the reader's keydown guard treats INPUT focus as typing — so arrow keys stepped the range input instead of turning pages. With step="any" each press moved ~1% of the book and key repeat crossed the whole book in seconds; the debounced progress saves then persisted every wild landing (positions scattered from page 5 to 285 to 100%, "progress no longer saves correctly after page turns"). Two layers: - reader.templ: all three progress sliders blur on pointerup, so a mouse scrub no longer leaves focus behind. - reader.ts handleKeydown: ArrowLeft/ArrowRight on a focused progress slider preventDefault, blur, and fall through to the page-turn handlers; every other key still belongs to the control, so a keyboard user who Tabbed to the slider keeps arrows for it. |
||
|
John O'Keefe
|
6791d611f0 |
fix(vite): guard cleanStaleChunks — vp test deleted every built chunk
The cleanStaleChunks plugin lives in the shared vite config, so its closeBundle hook also fired under `vp test` — where generateBundle never runs, produced stayed empty, and the sweep deleted every hashed chunk in web/static (28 files → 3). The entries still referenced their old imports, so every page 404'd and Alpine never started: the whole reader and media system appeared dead after merely running the test suite. Guard: skip the sweep when produced.size === 0 — not a build run, no build output to clean. Verified: build → 28 files, test → 28 files; back-to-back builds stay byte-identical. |
||
|
John O'Keefe
|
e5315f1b94 |
test(web): vitest suites for markdown, filters, password rules, api, storage, toast
Test infrastructure uses the vite-plus stack: vp test (bundled Vitest 5),
jsdom environment configured in vite.config.ts, colocated web/src/*.test.ts
files that tsc type-checks, vp fmt formats, and the bundler never imports.
- markdown.test.ts pins the notes XSS defenses: parser-level html escaping,
DOMPurify FORBID_TAGS, scheme checks, footnote rendering, html->markdown
round-trips incl. the headingless-table rule.
- filter_builder.test.ts pins the deep-link wire contract (ruleToFlatParam),
smart-serialization rules (isFlatable), and pruneIncomplete semantics.
- password_validation.test.ts covers the rules fetch + fallback and the
full submit-gating matrix through the real DOM flow (debounce included).
- api.test.ts pins response/error contracts incl. the HTTP-status and
Unknown-error fallbacks and the 404-as-null progress contract.
- storage.test.ts covers token/theme/library round-trips, the __all__
sentinel, and the SSR cookie.
- toast.test.ts pins config completeness for every ToastType (the missing
'warning' entry regression) and render behavior.
Found + fixed while writing these: the no-useless-escape fix in
|
||
|
John O'Keefe
|
86c4940426 |
style(toast): warning toast uses the /90 opacity variant
Release / build-and-push (push) Successful in 3m6s
Matches the error/success/info entries visually. bg-yellow-500/90 is generated by the Tailwind scanner from this file itself (content globs include web/src/**/*.ts), same as the sibling bg-*-500/90 classes — no safelist entry needed.v2.1.0 |
||
|
John O'Keefe
|
d7d1495753 |
fix(types): clear web/src type and lint debt with real types
tsc --noEmit and oxlint web/src are now both clean; vp check passes.
- htmx: one canonical Htmx interface in shims.d.ts (ajax/process/trigger
per the htmx 2.x API). alpine.ts had a second, conflicting Window.htmx
declaration (TS2717) — removed; bookPicker/collections lose their
(window as any) escapes now that Window.htmx is properly typed.
- tsconfig lib ES2020 -> ES2021 + DOM.Iterable: replaceAll exists
(escapeHtml) and for..of over querySelectorAll results is legal.
Type-checking only — vite emits at esnext regardless.
- custom-section-builder: selectedBooks holds identity fields only;
typed as Pick<BookInfo, ...> instead of forcing partial API shapes.
The API-critical BookInfo contract is untouched.
- toast: real bug found by the type system — setup.ts passes 'warning'
but getToastConfig had no entry, so the warning toast threw TypeError
at runtime instead of displaying. Added the type + config. Uses
bg-yellow-500 (verified present in scanned templates => generated by
Tailwind); the one intentional behavior change in this commit.
- book-detail: instanceof narrowing replaces an as-cast; unused
AnnotationSource import removed.
- password_validation: explicit HTMLElement | null for the checklist
list (querySelector('ul') infers HTMLUListElement, conflicting with
the container-as-list branch).
- stats: tooltip parsed value typed via annotation (bar=scalar,
line={x,y}); dead getActiveRange() removed (was already tree-shaken
out of every bundle).
- library-switcher: onSwitch honestly typed void | Promise<void>
(series.ts passes a sync navigator).
- 20 unused catch bindings converted to optional catch binding.
Verification: fresh build is byte-reproducible; every chunk is
identical to the pre-fix build except main.js (instanceof narrowing,
regex fix, stats refactor), reader.js (destructure), and the shared
storage chunk (renamed — carries toast.ts). reader/main diffs audited
hunk-by-hunk against the intended edits.
|
||
|
John O'Keefe
|
77cd268f4e |
style: format web/src with oxfmt defaults (via vp fmt)
One-time mechanical sweep, isolated from any functional change. - oxfmt defaults: double quotes, semicolons, 2-space indent, LF, printWidth 100, trailing commas, bracket spacing, arrow parens. - vite.config.ts gains an fmt.ignorePatterns block for web/static: vendored/minified assets (htmx.min.js, compiled css) must never be reformatted. - Verified: rebuild after the sweep produces byte-identical JS/CSS bundles; only sourcemaps changed (they embed sourcesContent). |
||
|
John O'Keefe
|
8aeecaa0ef |
Experiment: migrate web build to vite-plus 1.3.0
- vp migrate rewrote scripts (vite build -> vp build), aliased vite to @voidzero-dev/vite-plus-core 1.3.0 with an overrides pin, and switched the config import to vite-plus. - Manual one-line fix migrate missed: minify "esbuild" -> true (Vite 8 dropped bundled esbuild; oxc minifier is the replacement). - Widened cleanStaleChunks sweep from .js/.js.map to .css/.mjs too: Rolldown renames emitted css chunks, and two stale vite-7-era css files (annotation_layer_builder, text_layer_builder) survived the old js-only regex. - Verified: both custom plugins survive; prod/dev/watch builds OK; two consecutive builds idempotent at 289 files; vendor/pdfjs assets byte-identical; entry names (main.js/reader.js) unchanged. |
||
|
John O'Keefe
|
dd4cf1235c |
test: pin scan_extractor_version at the start of ev-sensitive tests
Release / build-and-push (push) Successful in 3m20s
The extractor/force-rescan tests assert manifest stamps against the default version (1), but the setting lives in the SHARED dev DB — any prior run (or a manual admin-UI bump) can leave it at 2, failing every stamp assertion through no fault of the code. Each ev-sensitive test now pins the setting to 1 at start and restores 1 on cleanup; proven against a deliberately bumped setting (tests pass, row restored).v2.0.0 |
||
|
John O'Keefe
|
a42c6d4f75 |
feat(scanner): force rescan = original scan; feat(filters): the manually-edited filter (faceted plan Phase 13)
Release / build-and-push (push) Successful in 3m7s
Force rescan contract change (owner-directed): the editorial-keep skip
is retired. Force rescan now re-derives EVERY field on EVERY book from
all sources — multi-file books included (the folder/zip flows ran the
full chain on create and on manifest-flagged edits, but skipped the
sidecar apply under force). metadata_overrides is the only protection:
ApplyMetadataOverrides re-applies manual values inside the writer, so
manual edits survive; everything else re-derives.
- A force pass also STAMPS each re-derived book's manifest with the
current parser version (success-only — same truthfulness rule as
Phase 10) and closes its open processing issues (Phase 11 self-heal).
Force rescan and the ev-bump trickle now converge: either leaves the
library fully current; the deploy order is bump-first, then Rescan
All (documented in the plan and contract_pins).
- Single-file force rescan already ran the full chain — unchanged,
now pinned.
NEW filter field manually_edited ("Manually edited", boolean):
cardinality(mi.metadata_overrides) > 0 / = 0. Composed-engine (qf)
only — the builder's Book facts group gains it with is true/false;
no value menu, no suggestion surface; family lens inherited.
Tests: multifile force applies sidecar metadata + stamps + override
protection; single-file pin; the filter both directions scoped to the
test library; adjacent suites (Audiobook, Calibre, ExtractorVersion,
CrossLibraryMove, ProcessingIssues, FilterBuilder, SchemaIdempotency)
green. The extractor mechanism itself is untouched — the owner bumps
scan_extractor_version manually via the admin UI (plan Phase 13 §3).
|
||
|
John O'Keefe
|
9b5defcb31 |
feat(issues): the honest processing-issues ledger — row-less import failures, self-healing resolution, family lens (faceted plan Phase 11)
processing_issues only recorded cross-library move rejections; files that failed to import vanished into an error counter and stdout. The ledger now records every failure class and heals itself: Schema (in place, pre-release boundary): - media_item_id NULLABLE + file_path TEXT (deliberately unbounded — an over-long path is itself a failure mode; a VARCHAR(500) ledger column died on the exact 22001 it existed to report — caught by the new test). Row-less failures key on (library, file_path); item-backed on (item, type). Widened dedup = COALESCE-expression unique index; the old table UNIQUE stays (NULLs pairwise-distinct for row-less). - Conflict-target upsert: repeat failures UPDATE the same row and REOPEN resolved ones — never duplicates. Wirings: - CreateMediaItem failure (single-file + multifile createAudioBookItem) → row-less import_failed (error). - Extraction hard-fail with fallback import → item-backed metadata_extraction_failed (warning). - Change-path re-extraction failure (the ev/sidecar guard) → item-backed metadata_extraction_failed; LogProcessingIssue keeps its signature, a row-less helper joins it. Auto-resolution: - Successful import → ResolveOpenProcessingIssuesForPath (library, file_path). - Successful (re)extraction → ResolveOpenProcessingIssuesForItem (single-file change path, multifile editorial apply). - Unchanged-skip scans do NOT resolve: extraction never re-ran, so the degraded metadata is still degraded — the page heals exactly when the book does. Family lens on ALL issues surfaces (list, stats, resolve, delete, SSR page): explicit out-of-set library id → 403, never silent. This tightens one pre-lens behavior — a nonexistent library id returned 200-empty, now 403 (pinned, changelog entry in docs/contract_pins.md). List is LEFT-JOINed so row-less rows appear (media_item_id "", title falls back to the path). Tests: full lifecycle state machine through real scans (fail → appears, repeat → no dups, success-at-path → resolved, LEFT-JOIN list shape); family lens 403s with a fresh non-shared admin; all 7 pre-existing issue suites green (two expectations widened to the lens, changelogged); adjacent suites (CrossLibraryMove, Audiobook, Calibre, ExtractorVersion, SchemaIdempotency) green; unit suites green. |
||
|
John O'Keefe
|
19b9c7c9b8 |
fix(tests): wire settings registry into the test auth handler — unstick TestAdminCreateUser
setupTestServer built the settings registry (and wired it into
systemSettingsHandler) but never called authHandler.SetSettings, unlike
main.go:112. The auth handler therefore ran with a nil registry and
defaultTimezone() took its nil-fallback branch, returning "UTC" for every
Register/AdminCreateUser in tests regardless of the system default_timezone
setting — so the
|
||
|
John O'Keefe
|
91b0c45dd4 |
feat(scanner): extractor-version (ev) mechanism — parser fixes reach old rows without a force rescan (faceted plan Phase 10)
Parser fixes (
|
||
|
John O'Keefe
|
43987262a6 |
fix(filters): operator table sent labels as wire values — every non-self-named operator was broken (user report)
The client's operator map passed the display string as BOTH the map key and the OpMeta.key, so the menu clicks transmitted 'doesn't contain' / 'isn't' / 'is empty' / 'at least' / … literally. The server rightly rejected them (unknown operator → 400), the count fetch's catch hid the counter, and flipping away from such an operator never came back. Only self-named ops (contains, is, has, between…) worked. - Map keys now DERIVE from meta.key (Object.fromEntries over the op list) — wire value and label structurally cannot diverge again. - New scripts/filter_builder_harness.js: a playwright harness that drives the real drawer in headless Chromium and sweeps every author operator, capturing filter-count responses and console errors — the first UI-level regression net for the builder (the server tests could never see a client-side mis-key). - Verified live: contains 8 → doesn't contain 60 → is 8 → isn't 60 → is empty 2 → refill 8; zero console errors across the sweep. |
||
|
John O'Keefe
|
68701d6b94 |
fix(filters): live count follows the top library picker (user report)
The count fetch reads the library select at fetch time, but nothing re-triggered it on a scope change — the number stayed stale (8 across all books) after switching to a library where the rules match nothing. The picker's change event now reschedules the count, same as rule edits do. |
||
|
John O'Keefe
|
ab713e2feb |
fix(filters): blank-typed browse 500 on id-valued fields (library/collection/added_by)
The browse-on-click blank sentinel was trimmed to empty (correct), but the typed query had already been bound unconditionally — the id-source suggestion query has no score CASE to reference it, leaving an unused parameter Postgres rejects (42P18). Every click of an empty library / collection / added-by box answered 'No matching values' via the error path. - ComposedFieldValues binds the typed query ONLY in branches that reference it and only when non-blank; the score expression is '0 AS score' for browse (no param) and the similarity CASE when typing. composedIDSourceValues takes the raw typed string and binds it only when the name filter is built. userBind already followed this discipline; audited every bind site in the file (fuzzy helpers, composed search, count) — all interpolate immediately. - Tests: library browse under rules (exactly the surviving library, count pinned), series browse blank, added_by browse blank (no error), collection browse blank (offered with count) and under a zero-match rule (never offered). Verified end-to-end against the live dev data: author+Sapkowski AND narrator+Kenny → Audiobooks (8). |
||
|
John O'Keefe
|
9c006a9a2f |
fix(filters): browse with no rules works, popup under the input, no sticky headers (user report)
- Blank typed queries rode the STATIC autocomplete path for the classic six fields, where a single space similarity-matches into nothing — clicking an empty value box with no other filters said 'No matching values' in a library full of values. Blank = browse now always rides the composed engine (which trims and answers the top values); static keeps its pinned contract for real typed queries. Regression-pinned: browse with no rules lists the field's values. - The value popup now drops BELOW the input (top: 100%), not over it. - Field-menu group headers lost position:sticky — the search box alone stays pinned. |
||
|
John O'Keefe
|
4c03116ec7 |
fix(filters): value menu flash-on-click, empty state, sticky field filter (user report)
- The flash was a race: on localhost the suggestion fetch resolved between the focus (mousedown) and the click event, so the just-opened menu was closed by its own click-outside handler — the input sits outside the menu div. Clicks anywhere inside the value cell now never close the menu; outside clicks do. - The menu opens immediately in a loading state; an empty browse renders 'No matching values' instead of nothing. Plain free-input fields (page_count etc.) open no menu at all. - Field menu: the filter input moved outside the scroll region — it sticks while the grouped list scrolls beneath it. |
||
|
John O'Keefe
|
c747f0a76c |
feat(filters): per-row AND/OR (Excel-style chips) + browse-on-click value menus (user report)
- Per-rule connectives: the chip before each rule flips that rule between AND and OR — OR-joined neighbors merge into an invisible OR-block, AND pulls them back out. The tree stays the single source of truth (implicit blocks are a client marker the server ignores), so deep links, saved filters, and the composed engine are untouched. Group-mode pills are gone; explicit groups remain as pure structure (+ group). Compile limit, same as Excel: flat chips express AND-of-OR-groups; a top-level OR of AND-blocks still needs nesting. - Browse-on-click: clicking an empty value box now fetches the field's top values UNDER the other rules (blank typed = browse, not a similarity filter) — 'assign an author, click Publisher, see their publishers' without typing. Server: the composed suggestion engine trims the typed query; client: the >=2-char guard no longer blocks the empty query, browse limit 15 vs 8 while typing. - Tests: the browse case pinned (blank sentinel + rules → the field's values under those rules, zero-match values absent). |
||
|
John O'Keefe
|
6f98ca4c22 |
fix(filters): group mode reads AND/OR, connective labels toggle it (user report)
The ALL/ANY pills required mapping ALL→AND, ANY→OR that nobody should have to learn. The pills now say AND / OR directly (wire values stay all|any — server contract untouched), and the AND/OR text between sibling rules became a clickable toggle for its parent group, so the switch lives exactly where you're looking. |
||
|
John O'Keefe
|
04b36f0563 |
fix(filters): builder UX round one — incomplete rules, group removal, field menu (user report)
- Root bug: incomplete rules (the ready rule, every new group's first rule) were sent to the server — the live-count fetch toasted a 400 and every suggestion fetch failed silently (why author typing showed no dropdown). Fix: completedTree() prunes incomplete rules and empty groups; the count fetch skips when nothing is complete, suggestion fetches send only the other COMPLETED rules, and serialization wires only completed rules (an unfinished rule was never a filter). - Group remove button was x-show'd away on depth-0 groups — exactly where user groups live. Always visible now. - Field menu: filter input at the top (Enter prevented), no-match state; group headers restyled as flush-left tinted sticky bands with options tabbed in beneath (user's layout pick). - toast.ts: filter-count fetches join the suggestion-fetch exemption (a failed background hint never interrupts). |
||
|
John O'Keefe
|
4ee24e7152 |
feat(filters): the rule builder — composed engines over the field registry (faceted plan Phase 9)
- Field registry (services/filter_registry.go): ~55 typed fields — every book column, file/lifecycle facts, the three HOLDING STRUCTURES (library, collections with excluded-aware membership, series with FULL primary+alternate_info membership), and the whole user-relative class (status, progress, last read, my rating, review, notes/highlights presence, note/highlight text, time spent, pages read, collections, added by) reading the VIEWER's rows. Operators are typed constants; negation is an operator. - Composed engines (services/filter_query.go): one Go clause builder over raw pgx feeds the search (identical SearchMediaItemsUnifiedRow shape), the field-aware suggestions, and a true-COUNT live count. Mandatory WHERE head everywhere (family lens + integrity facts). Values are always bound parameters — no user string reaches a field/operator position, ever. - Wire: the qf JSON param (validated strictly — unknown fields/operators/malformed values are 400); qf wins over flat params on hand-crafted URLs; flat deep links keep working unchanged. - Endpoints: GET /api/media-items/filter-count (live count); field-values autocomplete upgraded in place — any registry field key, qf binds the OTHER rules (symmetric under-filter suggestions, zero-match values never offered); classic six without a tree keep their static pinned contract. - UI: the drawer becomes the rule builder — the ONLY filter UI (templ markup, Alpine tree state, sanctioned island; grouped field menus, operator menus with red negations, field-aware value menus, live count, chips per rule with red negations, URL rehydration, smart serialization flat-when-simple else qf). - Tests: composed semantics (AND/OR/negation/nesting/enums/bools/ numbers/dates/empty), every user-relative field class with the my-data weld (two readers, different worlds), note-text substring pin (catharsis ≠ Carthage), family lens (hidden libraries nowhere, 403s), suggestion-under-rules exactness, qf validation + precedence, flat-vs-tree equivalence; registry completeness unit test. |
||
|
John O'Keefe
|
ebb7f4a5b0 |
feat(filters): panel removal + All-Books field-values fix (faceted plan Phase 8)
- Remove the Phase 7 facet-panel UI: SSR route, facet_panel.templ, FacetCountsForView, drawer panel block, panel TS + .facet-* CSS. The ENGINE stays (FacetCountsUnderFilters, all-facets endpoint, drift guard) — it becomes the Phase 9 builder's machinery. - Fix the All-Books bug: field-values autocomplete (author/genre/ tags/narrator/series/language) treats library_id as OPTIONAL — absent aggregates every VISIBLE library (family lens in-query); explicit out-of-set library_id is now 403, never a silent pass. The four scalar Search*Values queries get the nullable shape. - toast.ts: suggestion fetches are exempt from error toasts (console-only) — a failed background hint never interrupts. - TestFacetPanelPartial deleted with the behavior; TestFieldValuesAll- Libraries pins the fix (200 + cross-library aggregation, family lens, 403). Contract-pins changelog entry added. |
||
|
John O'Keefe
|
f52bb6f073 |
feat(facets): all-facets panel with under-filter counts (faceted plan Phase 7)
The 9 Batch A facets could not be SET from the web UI, the facets
endpoint had no in-app consumer, and the registry's libraryTypes hints
rendered nowhere. This phase closes all three.
- ONE UNION-ALL sqlc query (FacetCountsUnderFilters) computes all 14
listable facets in a single round trip: each arm omits its OWN
facet's filter block and carries every other registered facet's
block (copied mechanically from SearchMediaItemsUnified) — classic
faceted semantics: with filters active each facet counts under the
others; with none, blocks are neutral and the counts are the v1
global-per-library ones. One query serves both cases. The 14
per-column Batch C queries are deleted.
- Endpoint: no facet param → the new all-facets shape
({facets: {param: [{value, display?, count}]}}), filters bound via
the shared ParseMediaSearchParams, library_id optional (the
All-Libraries view, family lens still enforced); facet=X keeps the
pinned single-facet shape (now routed through the same query with
neutral filters).
- SSR partial GET /media-items/facet-panel: templ renders the drawer's
value lists — registry order, Batch D library-type scoping
(FacetsForLibraryType: manga_type on manga, reading_direction on
manga+comics, narrator on audiobooks; All view = union), prettified
genre labels + language display names, active values marked. Family
lens: out-of-set explicit library_id is 403.
- Drawer wiring: the panel fetches on drawer open and after every
apply with the CURRENT filter query string; a value click toggles
that facet's comma-joined filter (hidden input where the drawer has
no field — same as deep links) and re-applies the form; chips and
URL stay in sync. Values are server-rendered (rendering doctrine);
bookshelf.ts owns only the click/fetch orchestration.
- prettifyGenre moves to internal/utils as the canonical
implementation (services' display enrichment shares it; templ wraps
it; the TS port in bookshelf.ts is unchanged).
- Drift guard extended: FacetCountsUnderFiltersParams must carry every
registry facet's param, the bools as pgtype.Bool.
Tests: under-filter exactness (own-block omission, OR-within,
AND-across), family lens (hidden values absent; 403 on both shapes),
all-shape + pinned-shape coexistence, partial rendering with Batch D
scoping + active marking + 403. Full container suite green except the
documented pre-existing TestAdminCreateUser red.
|
||
|
John O'Keefe
|
6152137156 |
fix(tests): repair stale pins surfaced by the Phase 6 full-suite run
Two suite failures, neither caused by scanner code (the Phase 6 diff
touches no handler):
- search_test invalid_library_id expected the pre-family-lens 404;
|
||
|
John O'Keefe
|
2c5ad014c0 |
feat(scanner): multifile folder/zip sidecar detection (faceted plan Phase 6)
Reverses the Phase 4 scope cut: for EXISTING folder/zip audiobooks a
sidecar edit was invisible to everything, including force rescan —
updateAudioBookItem writes only scan-derived fields, and these formats
often have no useful embedded tags, so the sidecar is the only source.
- Change fingerprint gains the manifest compare in both processors:
changed := size differs || track payload differs || manifest drifted.
The ladder runs against the bulk-loaded map entry (stat pairs only,
zero extra queries); NULL stored manifest backfills without
re-extraction (deploy-day rule, same as single-file).
- Manifest-changed applies the once-extracted book metadata to the
editorial columns (updateMediaItemWithMetadata, overrides protected)
and re-runs the playlist hook. The apply runs BEFORE the focused
payload write: the generic update's format-group pass derives mime
from the sentinel (none) and chapters from metadata.ChapterCount
(unset) — the payload write must win (rescanMultiFileAudioBook's
ordering). Track-only changes keep the existing editorial-keep
update; force rescan keeps it too.
- Manifests seeded at multifile create (folder names derive from the
sentinel — the exact representative the extractor reads — via
detectSidecarChangeWithRepresentative, so manifest and extractor can
never disagree; a directory-basename cover is ignored by the
extractor and therefore not tracked).
- Zip parity fixes: the fast path now lifts archive state like the
folder path always did (a returned zip book stayed archived
forever), and the change branch handles the unchanged-archived case.
- Single-file sidecar change path: extract-failure guard — the old
non-nil empty &MediaMetadata{} fallback bypassed the update path's
filename-title guard (preExtracted == nil only) and would blank the
title on a failed extraction; the row is now left untouched and the
manifest refreshed. Empty-success extraction (sidecar removed from a
tag-less book) now passes nil so the create-equivalent chain (tags →
filename) re-derives instead of blanking.
- Multifile changes now count in the scan summary (changedItems);
they were previously uncounted.
Tests: sentinel-name derivation unit pin; folder sidecar edit
(title + collection sync + fresh manifest + steady-state skip),
track-only change keeps editorial, zip sidecar edit + archive
restore, sidecar removal falls back to filename (the wipe-guard's
testable sibling). Full container suite: green except the documented
pre-existing TestAdminCreateUser red.
|
||
|
John O'Keefe
|
5245f292ca |
feat(scanner): quiet scans, enqueue coalescing, watcher probe, sibling cache
Part 2 of the faceted plan, Phase 5 (steps 5-6): the scan that used to print ~4 lines per unchanged file now prints ONE summary line, and a poll tick landing mid-scan no longer stacks a second full scan. - Quiet scans (§4): the walk loop is silent — inserts, changes, and errors log inside processMediaFile; the duplicate 'Successfully processed' branch (both branches printed it) and the per-file 'processing / already exists / same size, skipping' chatter are gone. Summary carries the trigger and a changed counter: 'Scan of <folders> (trigger: poll): N files, N changed, N new, Xs, N errors'. Triggers threaded through every enqueue site: poll (startBackupScan), watcher (dirty-dirs flush), admin (scan API), initial (startup scan). - Enqueue coalescing (§5): Worker.HasActiveScanForLibrary (JobResult gains additive job_type so only SCAN jobs coalesce — an import sharing the library id never blocks one). enqueueLibraryScan skips silently; the admin scan API answers 409 with the active job id. Enforces the one-scan-per-library invariant the bulk scan-state map leans on. - Watcher capability probe (§4): ONE Add on a root folder; failure disables the watcher with a single startup line (replaces the per-directory warning spam on inotify-broken deployments). The watcher stays an optional latency reducer over scheduled scans. - Per-directory sibling cache (§6): hasSiblingBookFileCached caches the ReadDir per directory for one scan — loose-image manga re-stats thousands of images sharing a handful of directories. Tests: TestHasActiveScanForLibrary (unit — running/pending coalesce, terminal jobs don't, non-scan jobs don't, empty library never, nil worker safe). Full scanner suite green in the container. |
||
|
John O'Keefe
|
6750e9e0dc |
feat(scanner): manifest-gated sidecar change detection + playlist sync on change path
Part 2 of the faceted plan, Phase 4 (steps 3-4). Size-only detection
was blind to metadata-sidecar edits; a size-stable book is changed by
EDITING its metadata.opf/metadata.json/cover — previously invisible
until a force rescan.
- Sidecar manifest fingerprints (media_scanner_manifest.go): the
manifest stats exactly the names the extractors consume (metadata.opf,
metadata.json, findSidecarCover patterns incl. <basename>.<ext>), so
manifest and extractors can never disagree. Shape v1:
{v, entries:[{n, s, m (unix nanos — second granularity would hide
edits landing in the backfill scan's own second), h (sha256 — covers
included: the repoint ladder needs a baseline hash for any stat-
drifted sidecar; a hashless cover turns harmless mtime drift into a
re-extraction, the exact flood this design kills)]}.
- Evaluation ladder in cost order: stat pairs identical → skip with
ZERO bytes read; stat drift → hash that sidecar (equal = host repoint
reconciliation, stat pair refreshed, no re-extraction; differing =
changed); NULL stored manifest → backfill only, NO re-extraction
(deploy-day rule: the first scan after release must not look like an
mtime flood).
- Change path: re-read full row, extractMetadata once and pass it
through updateMediaItemWithMetadata (preExtracted — metadata_overrides
stay protected), run syncPlaylistCollectionsAfterCreate (Correction 2:
a reading-order entry added to an EXISTING book now creates its
collection; add-if-missing is idempotent), store a fresh full
manifest. Creates seed their manifest at insert time.
- Multifile folder/zip books intentionally keep Phase 3 behavior: their
change path does not consume sidecar extractors, so gating them would
flip books 'changed' without picking anything up (plan correction —
the plan's §3 multifile note assumed otherwise).
- UpsertMediaScanState + GetMediaScanState queries.
Tests: media_scanner_manifest_test.go (ladder unit tests — fast path,
repoint, same-size drift, appear/vanish, unhashable legacy entries);
TestScanManifestLifecycle (seeding, unchanged-skip stability, repoint
reconciliation with stat-pair refresh assertion, same-size edit
consumed, playlist sync on the change path + idempotency, no-leak
guard). Container runner.
|
||
|
John O'Keefe
|
693053a9f5 |
perf(scanner): bulk scan-state load — one query per library scan
Part 2 of the faceted plan, Phase 3 (steps 1-2). The periodic scan
issued one GetMediaItemByFilePath per scannable file (~3,094 queries
per scan in prod, nearly all answering 'unchanged, skip') plus one
GetLibraryByFolder per file for folder resolution.
- NEW media_scan_state table: the scanner's per-item fingerprint
cache. Deliberately NOT a media_items column — the library-scoped
list emits raw sqlc rows as JSON (
|
||
|
John O'Keefe
|
bbedbfb13f |
feat(facets+ui): clickable facet values everywhere, genre prettify, active-filter chips
Frontend half of the faceted-navigation Phase 2. 'Everything filterable should be clickable': every displayed facet value links to the bookshelf pre-filtered by its raw stored value; display forms are prettified at render, never in the DB. - templates/facet_links.templ: FacetLink (inline links) + FacetBadge (badge links) to /bookshelf?<facet>_filter=<value>; the raw value rides the param while the shown text is prettified, so matching and display can never disagree. Series/tags keep their dedicated pages by design; the card media-type icon stays unlinked (a derived kind, not a column value — deep-linking it would send wrong values). - book_detail.templ: publisher, language, edition, genre, imprint cells + narrators line + age-rating/story-arc badges are now links. - prettifyGenre (templates/utils.go + unit tests): title-cases only entirely-lowercase values; small words stay down except first position; & preserved. TS port for chip labels documents the twin. - bookshelf.templ + bookshelf.ts: active-filter chips bar — Alpine renders chips from client-local form state (sanctioned island); one bubbled htmx:afterRequest hook keeps chips + shareable URL in sync; removeFilterChip clears the facet and re-applies. URL deep links now accept all 18 facet params (hidden inputs where the drawer has no field). Pins changelog: Batch B/C entry in docs/contract_pins.md. Rendering ownership guard (make lint-rendering) green. |
||
|
John O'Keefe
|
7e586df48a |
feat(facets): facet registry, Batch A server params, genre model v2, facets endpoint
Phase 1 + backend of Phase 2 of the faceted-navigation plan. One
declarative facet registry (services/facet_registry.go) drives the
bookshelf filter system; a drift-guard unit test reflects over the
generated SearchMediaItemsUnifiedParams so a registry entry and its
sqlc.narg block can never disagree silently.
- Nine new filter params on SearchMediaItemsUnified: story_arc,
imprint, edition (scalar-fuzzy), age_rating, format_group,
manga_type, reading_direction (exact), explicit, abridged
(presence-bound bools). Comma = OR-within multi-select on every
text facet; AND-across-facets unchanged.
- Genre model v2 (user rule: a subgenre is not a genre, it's a tag):
genre_lexicon.go rebuilt as shelf set + subgenre->shelf map;
pickSmartGenre picks direct shelf > mapped subgenre > Fiction floor
> empty (unknown-only stays tags-only). Type-words (Manga, Comics,
Graphic Novel) join the demote list. Both 'adventure' and 'action &
adventure' are shelves — provider data uses both and the verified
traces pin them.
- Language filter UX: comma elements normalize through the ingest
language map, so display names ('English') match rows stored as ISO
codes ('en'). The language autocomplete now serves display names.
- NEW GET /api/media-items/facets?facet=X&library_id=... — registry-
driven value+count listing (14 static per-column queries, family-lens
join); language values carry display names; bool/range facets 400.
- Family-lens gates: an explicit hidden-library library_id is now a
403 on the search endpoints (was a silent empty set) — JSON branch
and HTMX grid partial both.
- handlers.ParseMediaSearchParams is the SINGLE param parser for the
search JSON endpoint and the HTMX grid partial; the HTML branch
previously dropped publisher_filter and narrator_filter silently.
Pins: docs/contract_pins.md changelog entries (additive request
params; new facets endpoint). Tests: facet_registry_test.go (shape +
SQL drift guard + library-type union), series_parsing_test.go genre-v2
cases, facets_integration_test.go (param filtering e2e, counts,
language display, 403s) — container runner.
|
||
|
John O'Keefe
|
7537063cb7 |
feat(filters+nav): author navigation end-to-end — arrays on every media response, deep-linkable bookshelf filters, clickable authors
Completes the A2c acceptance scenario: click an author on a book detail page -> land on the bookshelf filtered to everything that author wrote (including their share of co-authored books). - Media detail + unscoped list resolved maps expose subtitle, explicit, authors, publishers (the library-scoped raw branch and the search endpoint already did — raw rows follow the SELECT). - SearchMediaItemsUnified SELECT carries the new columns; the search endpoint serializes the row directly, so clients get them there too. - book_detail.templ renders each author as a link to /bookshelf?author_filter=<name>. - Bookshelf honors filter params in the URL on load (author_filter, series_filter, genre_filter, publisher_filter, tags_filter, narrator_filter, language_filter): pre-fills the form, applies it to the first grid load. Comma-joined values are multi-select via the server-side string_to_array split. Filtered views are shareable. Pins widened: search rows (+7 keys), resolved keys (+4); changelog entries in docs/contract_pins.md. Android follow-up plan written to ~/.opencode/plan/bookhoard-app-v2-compat.md. |
||
|
John O'Keefe
|
f94e7472b4 |
feat(filters+ui): per-element author/publisher filters, playlist toggles, Up Next API, reorder UI, dashboard badges
Phases 4 + 5 of the metadata overhaul (plan §4, §5) plus §3 handler wiring. Filters (§4.1): author_filter/publisher_filter now match per-element against the authors[]/publishers[] arrays — a co-authored book matches any one of its authors — with comma-joined params acting as OR-within-facet multi-select (string_to_array + cross unnest). Legacy single-value params unchanged on the wire; series/genre/language filters stay on their varchar columns; narrator filter already per-element. Collections API (§3.3-3.5): collection detail carries viewer + hide_finished into the listing query and exposes up_next (visible- library-enforced reading-queue head); PUT /api/collections/:id/items/order (owner-only, full-order rewrite); UpdateCollectionRequest gains hide_finished/show_up_next ([] string last-value-wins for the hidden-false-sibling checkbox pattern) merged server-side OVER stored view_settings so imported_from_series provenance survives edits; EditCollectionModal renders the toggles from view_settings. Dashboard (§5.5): sections JSON carries is_playlist/complete (dashboard.ts is the AGENTS-documented JSON-render exception; first paint stays templ). Fully-read playlists render '✓ Playlist complete' instead of an empty rail; imported playlists get a 📖 Playlist badge. Frontend: book detail shows subtitle + full authors + Explicit badge; edit modal gains subtitle, comma-separated authors/publishers (client- split to arrays exactly like narrators), step=any series number, explicit checkbox; collection book cards get playlist move ↑/↓ buttons persisting the full order via the reorder endpoint (templ partial re-render on failure — DOM order is client-local, API order is truth). AGENTS.md gotcha #3 amended per plan §3.7: user collections are dashboard sections by default; the show_on_dashboard premise is retired. Verified: full affected integration sweep green (pins widened for up_next/is_playlist/complete — changelog entries in docs/contract_pins.md); unit suite green; go build + templ + vite green. |
||
|
John O'Keefe
|
1dbb0ce406 |
feat(collections): playlist auto-creation, position ordering, hide_finished, Up Next, dashboard eligibility
Phase 2b + 3 of the metadata overhaul (plan §2.4, §3.1-3.7). Auto-playlist collections (§2.4): the scanner turns obviously-playlist sidecar series (explicit '(Reading Order)'/chronological/publication/ in-order/timeline markers only — precision over recall) into curated static collections: upsert by view_settings.imported_from_series then name (rename-safe), position set ONLY on insert (clean integers; 1.5/1a stay unpositioned rather than guessing), membership add-if-missing, never removed or re-ordered by rescans, provenance in description + view_settings, show_on_dashboard true (dashboard-eligible by design), 'scanner.auto_playlist_collections' setting (default on, Settings Registry + schema seed). Ordering (§3.1-3.2): collection listings order position ASC NULLS LAST then added_at ASC (was added_at DESC — walked playlists backwards); PUT /api/collections/:id/items/order applies full-order rewrites (owner-only, service-gated). hide_finished + Up Next (§3.3-3.4): GetCollectionItemsForDashboard gains viewer_user_id + hide_finished (LEFT JOIN reading_progress, COALESCE percentage < 0.999) and viewer_percentage rides the row for card graying; GetCollectionUpNext serves the collection's reading-queue head with the family-lens visible-library array enforced. Dashboard (§3.5): user collections are dashboard-ELIGIBLE BY DEFAULT — the show_on_dashboard gate (never writable for user rows since the column shipped) is gone from GetUserCollectionsForDashboard; hiding stays dashboard-prefs (hidden_collections) territory; hide_finished filtered-everything now emits a Complete section instead of silently vanishing; sections JSON carries is_playlist/complete. Series page ordering hardening (§3.6): GetSeriesBooks tiebreaks by title ASC — NULL/tied numbers previously returned heap (scan) order, which is what the user saw. Frontend custom-section-builder note: the builder has sent show_on_dashboard: true since its creation; the API never bound it. User-row visibility is now intentionally prefs-governed, so the field stays unbound by design (documented in code). Pins widened (docs/contract_pins.md): collection detail + up_next; dashboard sections + is_playlist/complete; new reorder endpoint recorded. Verified: TestPlaylistCollectionsEndToEnd (new, full e2e: sidecar series split, collection auto-creation with position 7, rescan idempotency, owner-only reorder, finished-filter + completion marker + Up Next drain) plus the full affected integration sweep green. |
||
|
John O'Keefe
|
7e4c97f639 |
feat(scanner): fix ABS sidecar series parsing; three-way series mapping; smart genre; language codes; title-derived series
Phase 2a of the metadata overhaul (plan §2.1-2.3, §2.5-2.9). The headline bug: ABS writes sidecar series as an array of STRINGS (["The Witcher #3", ...]); the parser only accepted {series, sequence} OBJECT arrays, so every ABS-matched book silently lost its series (found live: 9 sidecars, zero series ingested; only folder-structure Malazan survived). parseSeriesCandidates now normalizes string arrays, object arrays and Libation plain strings. Three-way series assignment (plan §2.2): - primary bibliographic series -> series/series_number columns - additional real series (Drizzt: Icewind Dale #1 = Legend of Drizzt #4) -> alternate_info JSONB, comic-established shape - ordering constructs ('The Witcher (Reading Order)') -> PlaylistSeries candidates; NEVER the series column. Detection is explicit-marker-only ((reading order|chronological|publication order|in order|timeline) as parenthetical/suffix) — precision over recall; a series merely containing 'timeline' mid-name stays a series. Sequence policy: leading-number extraction ('Book 2' -> 2, '2, Dramatized Adaptation' -> 2, '1a' -> 1); lossy extractions preserve the raw text in alternate_info.series_sequence_raw. Smart genre (deterministic lexicon): compound Audible blobs split ('Epic, Action & Adventure, Military, Sword & Sorcery' -> 4 candidates), alias-normalized, format noise demoted ('Audiobook' never wins), most-specific canonical genre takes the genre column, all candidates stay in tags. Language: display names normalize to ISO 639-1 on ingest; ROOT-CAUSED the update-path drop — CreateMediaItem call site never set Language (explicit NULL over the column default), the update path did; create now carries it. Title-derived series fallback (§2.7): 'Baptism of Fire: The Witcher, Book 3' 2-level folders split conservatively (only when the post-colon tail matches ', Book N'), subtitle parsed for series+number, blanks-only precedence, matching-name number fill, zero display mutation. Works with no sidecar and no network. Ingest fixes: subtitle/explicit land in their columns (audio tag subtitle stops joining into title); full authors[] array (was [0]- only); sidecar descriptions sanitized via bluemonday UGC policy (vendored). metadata_overrides: verified scan-update path already applies ApplyMetadataOverrides; subtitle/explicit/authors covered by the Phase 1 override adaptation. Tests: new series_parsing_test.go (string-array sidecar e2e, Drizzt alternate_info, lossy raw preservation, playlist classification, title-derived fallback incl. no-mutation + blanks-only, genre lexicon, language map); audio-tag subtitle pin updated to column behavior; calibre/ABS 2.5-series_index pins widened per float change (changelog'd in phase 1). Deferred to 2b: auto-created playlist collections (§2.4) — settings toggle, collection upsert, ownership. |
||
|
John O'Keefe
|
646477afad |
feat(schema): subtitle/explicit columns, authors+publishers arrays, float series numbers
Phase 1 of the series/collections/metadata overhaul
(~/.opencode/plan/series-collections-metadata-overhaul.md).
Schema (in place, idempotent):
- media_items.series_number INTEGER -> DOUBLE PRECISION: real-world
ordinals are decimals (0.5, 1.5 novellas); INTEGER silently truncated
them (calibre 2.5 pinned as 2 in tests).
- + subtitle TEXT, explicit BOOLEAN (sidecar ingest; ABS explicit /
Audible isAdult).
- + authors/authors_search/publishers/publishers_search TEXT[] NOT NULL
(narrators A2a precedent). Arrays are write truth; author/publisher
varchars become trigger-regenerated display strings
(sync_author_publisher_display) so every varchar consumer (trgm
search, facets, API, OPDS/Kobo) keeps working unchanged.
- Row-level NULL normalization for the new arrays in
set_library_type_name (A2 lesson: Go nil -> explicit NULL -> 23502).
- Backfills ARRAY[author]/ARRAY[publisher] — deliberately no
comma-splitting ('Tolkien, J.R.R.' would corrupt; scanner kept only
authors[0] historically).
- collection_items.position INTEGER + (collection_id, position) index
for playlist ordering.
Go: sqlc regen; MediaMetadata/handler params gain Subtitle/Explicit/
Authors/Publishers (legacy singles still accepted, arrays win);
SeriesNumber int32 -> float64 end to end; metadata_overrides adapted
with legacy override keys ('author'/'publisher') preserved so existing
user overrides survive; bulk author edit writes the array.
Pins widened (docs/contract_pins.md): raw sqlc row key set + the six
new columns; changelog entry added.
Verified: unit suite green; integration sweep (bulk/smart collections/
scanner/stats/sync/queue/calibre/pins/audiobooks) green against the
shared dev DB after in-place migration. Pre-existing unrelated red:
TestOwnershipAndAdminTestSuite/TestAdminCreateUser timezone inheritance
fails identically at the pre-change commit (live shared state).
|