d7d14957537aba68a7ff9d9c718b222b7274be7c
1954
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 John O'Keefe
|
d7d1495753 |
fix(types): clear web/src type and lint debt with real types
tsc --noEmit and oxlint web/src are now both clean; vp check passes.
- htmx: one canonical Htmx interface in shims.d.ts (ajax/process/trigger
per the htmx 2.x API). alpine.ts had a second, conflicting Window.htmx
declaration (TS2717) — removed; bookPicker/collections lose their
(window as any) escapes now that Window.htmx is properly typed.
- tsconfig lib ES2020 -> ES2021 + DOM.Iterable: replaceAll exists
(escapeHtml) and for..of over querySelectorAll results is legal.
Type-checking only — vite emits at esnext regardless.
- custom-section-builder: selectedBooks holds identity fields only;
typed as Pick<BookInfo, ...> instead of forcing partial API shapes.
The API-critical BookInfo contract is untouched.
- toast: real bug found by the type system — setup.ts passes 'warning'
but getToastConfig had no entry, so the warning toast threw TypeError
at runtime instead of displaying. Added the type + config. Uses
bg-yellow-500 (verified present in scanned templates => generated by
Tailwind); the one intentional behavior change in this commit.
- book-detail: instanceof narrowing replaces an as-cast; unused
AnnotationSource import removed.
- password_validation: explicit HTMLElement | null for the checklist
list (querySelector('ul') infers HTMLUListElement, conflicting with
the container-as-list branch).
- stats: tooltip parsed value typed via annotation (bar=scalar,
line={x,y}); dead getActiveRange() removed (was already tree-shaken
out of every bundle).
- library-switcher: onSwitch honestly typed void | Promise<void>
(series.ts passes a sync navigator).
- 20 unused catch bindings converted to optional catch binding.
Verification: fresh build is byte-reproducible; every chunk is
identical to the pre-fix build except main.js (instanceof narrowing,
regex fix, stats refactor), reader.js (destructure), and the shared
storage chunk (renamed — carries toast.ts). reader/main diffs audited
hunk-by-hunk against the intended edits.
|
||
|
John O'Keefe
|
77cd268f4e |
style: format web/src with oxfmt defaults (via vp fmt)
One-time mechanical sweep, isolated from any functional change. - oxfmt defaults: double quotes, semicolons, 2-space indent, LF, printWidth 100, trailing commas, bracket spacing, arrow parens. - vite.config.ts gains an fmt.ignorePatterns block for web/static: vendored/minified assets (htmx.min.js, compiled css) must never be reformatted. - Verified: rebuild after the sweep produces byte-identical JS/CSS bundles; only sourcemaps changed (they embed sourcesContent). |
||
|
John O'Keefe
|
8aeecaa0ef |
Experiment: migrate web build to vite-plus 1.3.0
- vp migrate rewrote scripts (vite build -> vp build), aliased vite to @voidzero-dev/vite-plus-core 1.3.0 with an overrides pin, and switched the config import to vite-plus. - Manual one-line fix migrate missed: minify "esbuild" -> true (Vite 8 dropped bundled esbuild; oxc minifier is the replacement). - Widened cleanStaleChunks sweep from .js/.js.map to .css/.mjs too: Rolldown renames emitted css chunks, and two stale vite-7-era css files (annotation_layer_builder, text_layer_builder) survived the old js-only regex. - Verified: both custom plugins survive; prod/dev/watch builds OK; two consecutive builds idempotent at 289 files; vendor/pdfjs assets byte-identical; entry names (main.js/reader.js) unchanged. |
||
|
John O'Keefe
|
dd4cf1235c |
test: pin scan_extractor_version at the start of ev-sensitive tests
Release / build-and-push (push) Successful in 3m20s
The extractor/force-rescan tests assert manifest stamps against the default version (1), but the setting lives in the SHARED dev DB — any prior run (or a manual admin-UI bump) can leave it at 2, failing every stamp assertion through no fault of the code. Each ev-sensitive test now pins the setting to 1 at start and restores 1 on cleanup; proven against a deliberately bumped setting (tests pass, row restored).v2.0.0 |
||
|
John O'Keefe
|
a42c6d4f75 |
feat(scanner): force rescan = original scan; feat(filters): the manually-edited filter (faceted plan Phase 13)
Release / build-and-push (push) Successful in 3m7s
Force rescan contract change (owner-directed): the editorial-keep skip
is retired. Force rescan now re-derives EVERY field on EVERY book from
all sources — multi-file books included (the folder/zip flows ran the
full chain on create and on manifest-flagged edits, but skipped the
sidecar apply under force). metadata_overrides is the only protection:
ApplyMetadataOverrides re-applies manual values inside the writer, so
manual edits survive; everything else re-derives.
- A force pass also STAMPS each re-derived book's manifest with the
current parser version (success-only — same truthfulness rule as
Phase 10) and closes its open processing issues (Phase 11 self-heal).
Force rescan and the ev-bump trickle now converge: either leaves the
library fully current; the deploy order is bump-first, then Rescan
All (documented in the plan and contract_pins).
- Single-file force rescan already ran the full chain — unchanged,
now pinned.
NEW filter field manually_edited ("Manually edited", boolean):
cardinality(mi.metadata_overrides) > 0 / = 0. Composed-engine (qf)
only — the builder's Book facts group gains it with is true/false;
no value menu, no suggestion surface; family lens inherited.
Tests: multifile force applies sidecar metadata + stamps + override
protection; single-file pin; the filter both directions scoped to the
test library; adjacent suites (Audiobook, Calibre, ExtractorVersion,
CrossLibraryMove, ProcessingIssues, FilterBuilder, SchemaIdempotency)
green. The extractor mechanism itself is untouched — the owner bumps
scan_extractor_version manually via the admin UI (plan Phase 13 §3).
|
||
|
John O'Keefe
|
9b5defcb31 |
feat(issues): the honest processing-issues ledger — row-less import failures, self-healing resolution, family lens (faceted plan Phase 11)
processing_issues only recorded cross-library move rejections; files that failed to import vanished into an error counter and stdout. The ledger now records every failure class and heals itself: Schema (in place, pre-release boundary): - media_item_id NULLABLE + file_path TEXT (deliberately unbounded — an over-long path is itself a failure mode; a VARCHAR(500) ledger column died on the exact 22001 it existed to report — caught by the new test). Row-less failures key on (library, file_path); item-backed on (item, type). Widened dedup = COALESCE-expression unique index; the old table UNIQUE stays (NULLs pairwise-distinct for row-less). - Conflict-target upsert: repeat failures UPDATE the same row and REOPEN resolved ones — never duplicates. Wirings: - CreateMediaItem failure (single-file + multifile createAudioBookItem) → row-less import_failed (error). - Extraction hard-fail with fallback import → item-backed metadata_extraction_failed (warning). - Change-path re-extraction failure (the ev/sidecar guard) → item-backed metadata_extraction_failed; LogProcessingIssue keeps its signature, a row-less helper joins it. Auto-resolution: - Successful import → ResolveOpenProcessingIssuesForPath (library, file_path). - Successful (re)extraction → ResolveOpenProcessingIssuesForItem (single-file change path, multifile editorial apply). - Unchanged-skip scans do NOT resolve: extraction never re-ran, so the degraded metadata is still degraded — the page heals exactly when the book does. Family lens on ALL issues surfaces (list, stats, resolve, delete, SSR page): explicit out-of-set library id → 403, never silent. This tightens one pre-lens behavior — a nonexistent library id returned 200-empty, now 403 (pinned, changelog entry in docs/contract_pins.md). List is LEFT-JOINed so row-less rows appear (media_item_id "", title falls back to the path). Tests: full lifecycle state machine through real scans (fail → appears, repeat → no dups, success-at-path → resolved, LEFT-JOIN list shape); family lens 403s with a fresh non-shared admin; all 7 pre-existing issue suites green (two expectations widened to the lens, changelogged); adjacent suites (CrossLibraryMove, Audiobook, Calibre, ExtractorVersion, SchemaIdempotency) green; unit suites green. |
||
|
John O'Keefe
|
19b9c7c9b8 |
fix(tests): wire settings registry into the test auth handler — unstick TestAdminCreateUser
setupTestServer built the settings registry (and wired it into
systemSettingsHandler) but never called authHandler.SetSettings, unlike
main.go:112. The auth handler therefore ran with a nil registry and
defaultTimezone() took its nil-fallback branch, returning "UTC" for every
Register/AdminCreateUser in tests regardless of the system default_timezone
setting — so the
|
||
|
John O'Keefe
|
91b0c45dd4 |
feat(scanner): extractor-version (ev) mechanism — parser fixes reach old rows without a force rescan (faceted plan Phase 10)
Parser fixes (
|
||
|
John O'Keefe
|
43987262a6 |
fix(filters): operator table sent labels as wire values — every non-self-named operator was broken (user report)
The client's operator map passed the display string as BOTH the map key and the OpMeta.key, so the menu clicks transmitted 'doesn't contain' / 'isn't' / 'is empty' / 'at least' / … literally. The server rightly rejected them (unknown operator → 400), the count fetch's catch hid the counter, and flipping away from such an operator never came back. Only self-named ops (contains, is, has, between…) worked. - Map keys now DERIVE from meta.key (Object.fromEntries over the op list) — wire value and label structurally cannot diverge again. - New scripts/filter_builder_harness.js: a playwright harness that drives the real drawer in headless Chromium and sweeps every author operator, capturing filter-count responses and console errors — the first UI-level regression net for the builder (the server tests could never see a client-side mis-key). - Verified live: contains 8 → doesn't contain 60 → is 8 → isn't 60 → is empty 2 → refill 8; zero console errors across the sweep. |
||
|
John O'Keefe
|
68701d6b94 |
fix(filters): live count follows the top library picker (user report)
The count fetch reads the library select at fetch time, but nothing re-triggered it on a scope change — the number stayed stale (8 across all books) after switching to a library where the rules match nothing. The picker's change event now reschedules the count, same as rule edits do. |
||
|
John O'Keefe
|
ab713e2feb |
fix(filters): blank-typed browse 500 on id-valued fields (library/collection/added_by)
The browse-on-click blank sentinel was trimmed to empty (correct), but the typed query had already been bound unconditionally — the id-source suggestion query has no score CASE to reference it, leaving an unused parameter Postgres rejects (42P18). Every click of an empty library / collection / added-by box answered 'No matching values' via the error path. - ComposedFieldValues binds the typed query ONLY in branches that reference it and only when non-blank; the score expression is '0 AS score' for browse (no param) and the similarity CASE when typing. composedIDSourceValues takes the raw typed string and binds it only when the name filter is built. userBind already followed this discipline; audited every bind site in the file (fuzzy helpers, composed search, count) — all interpolate immediately. - Tests: library browse under rules (exactly the surviving library, count pinned), series browse blank, added_by browse blank (no error), collection browse blank (offered with count) and under a zero-match rule (never offered). Verified end-to-end against the live dev data: author+Sapkowski AND narrator+Kenny → Audiobooks (8). |
||
|
John O'Keefe
|
9c006a9a2f |
fix(filters): browse with no rules works, popup under the input, no sticky headers (user report)
- Blank typed queries rode the STATIC autocomplete path for the classic six fields, where a single space similarity-matches into nothing — clicking an empty value box with no other filters said 'No matching values' in a library full of values. Blank = browse now always rides the composed engine (which trims and answers the top values); static keeps its pinned contract for real typed queries. Regression-pinned: browse with no rules lists the field's values. - The value popup now drops BELOW the input (top: 100%), not over it. - Field-menu group headers lost position:sticky — the search box alone stays pinned. |
||
|
John O'Keefe
|
4c03116ec7 |
fix(filters): value menu flash-on-click, empty state, sticky field filter (user report)
- The flash was a race: on localhost the suggestion fetch resolved between the focus (mousedown) and the click event, so the just-opened menu was closed by its own click-outside handler — the input sits outside the menu div. Clicks anywhere inside the value cell now never close the menu; outside clicks do. - The menu opens immediately in a loading state; an empty browse renders 'No matching values' instead of nothing. Plain free-input fields (page_count etc.) open no menu at all. - Field menu: the filter input moved outside the scroll region — it sticks while the grouped list scrolls beneath it. |
||
|
John O'Keefe
|
c747f0a76c |
feat(filters): per-row AND/OR (Excel-style chips) + browse-on-click value menus (user report)
- Per-rule connectives: the chip before each rule flips that rule between AND and OR — OR-joined neighbors merge into an invisible OR-block, AND pulls them back out. The tree stays the single source of truth (implicit blocks are a client marker the server ignores), so deep links, saved filters, and the composed engine are untouched. Group-mode pills are gone; explicit groups remain as pure structure (+ group). Compile limit, same as Excel: flat chips express AND-of-OR-groups; a top-level OR of AND-blocks still needs nesting. - Browse-on-click: clicking an empty value box now fetches the field's top values UNDER the other rules (blank typed = browse, not a similarity filter) — 'assign an author, click Publisher, see their publishers' without typing. Server: the composed suggestion engine trims the typed query; client: the >=2-char guard no longer blocks the empty query, browse limit 15 vs 8 while typing. - Tests: the browse case pinned (blank sentinel + rules → the field's values under those rules, zero-match values absent). |
||
|
John O'Keefe
|
6f98ca4c22 |
fix(filters): group mode reads AND/OR, connective labels toggle it (user report)
The ALL/ANY pills required mapping ALL→AND, ANY→OR that nobody should have to learn. The pills now say AND / OR directly (wire values stay all|any — server contract untouched), and the AND/OR text between sibling rules became a clickable toggle for its parent group, so the switch lives exactly where you're looking. |
||
|
John O'Keefe
|
04b36f0563 |
fix(filters): builder UX round one — incomplete rules, group removal, field menu (user report)
- Root bug: incomplete rules (the ready rule, every new group's first rule) were sent to the server — the live-count fetch toasted a 400 and every suggestion fetch failed silently (why author typing showed no dropdown). Fix: completedTree() prunes incomplete rules and empty groups; the count fetch skips when nothing is complete, suggestion fetches send only the other COMPLETED rules, and serialization wires only completed rules (an unfinished rule was never a filter). - Group remove button was x-show'd away on depth-0 groups — exactly where user groups live. Always visible now. - Field menu: filter input at the top (Enter prevented), no-match state; group headers restyled as flush-left tinted sticky bands with options tabbed in beneath (user's layout pick). - toast.ts: filter-count fetches join the suggestion-fetch exemption (a failed background hint never interrupts). |
||
|
John O'Keefe
|
4ee24e7152 |
feat(filters): the rule builder — composed engines over the field registry (faceted plan Phase 9)
- Field registry (services/filter_registry.go): ~55 typed fields — every book column, file/lifecycle facts, the three HOLDING STRUCTURES (library, collections with excluded-aware membership, series with FULL primary+alternate_info membership), and the whole user-relative class (status, progress, last read, my rating, review, notes/highlights presence, note/highlight text, time spent, pages read, collections, added by) reading the VIEWER's rows. Operators are typed constants; negation is an operator. - Composed engines (services/filter_query.go): one Go clause builder over raw pgx feeds the search (identical SearchMediaItemsUnifiedRow shape), the field-aware suggestions, and a true-COUNT live count. Mandatory WHERE head everywhere (family lens + integrity facts). Values are always bound parameters — no user string reaches a field/operator position, ever. - Wire: the qf JSON param (validated strictly — unknown fields/operators/malformed values are 400); qf wins over flat params on hand-crafted URLs; flat deep links keep working unchanged. - Endpoints: GET /api/media-items/filter-count (live count); field-values autocomplete upgraded in place — any registry field key, qf binds the OTHER rules (symmetric under-filter suggestions, zero-match values never offered); classic six without a tree keep their static pinned contract. - UI: the drawer becomes the rule builder — the ONLY filter UI (templ markup, Alpine tree state, sanctioned island; grouped field menus, operator menus with red negations, field-aware value menus, live count, chips per rule with red negations, URL rehydration, smart serialization flat-when-simple else qf). - Tests: composed semantics (AND/OR/negation/nesting/enums/bools/ numbers/dates/empty), every user-relative field class with the my-data weld (two readers, different worlds), note-text substring pin (catharsis ≠ Carthage), family lens (hidden libraries nowhere, 403s), suggestion-under-rules exactness, qf validation + precedence, flat-vs-tree equivalence; registry completeness unit test. |
||
|
John O'Keefe
|
ebb7f4a5b0 |
feat(filters): panel removal + All-Books field-values fix (faceted plan Phase 8)
- Remove the Phase 7 facet-panel UI: SSR route, facet_panel.templ, FacetCountsForView, drawer panel block, panel TS + .facet-* CSS. The ENGINE stays (FacetCountsUnderFilters, all-facets endpoint, drift guard) — it becomes the Phase 9 builder's machinery. - Fix the All-Books bug: field-values autocomplete (author/genre/ tags/narrator/series/language) treats library_id as OPTIONAL — absent aggregates every VISIBLE library (family lens in-query); explicit out-of-set library_id is now 403, never a silent pass. The four scalar Search*Values queries get the nullable shape. - toast.ts: suggestion fetches are exempt from error toasts (console-only) — a failed background hint never interrupts. - TestFacetPanelPartial deleted with the behavior; TestFieldValuesAll- Libraries pins the fix (200 + cross-library aggregation, family lens, 403). Contract-pins changelog entry added. |
||
|
John O'Keefe
|
f52bb6f073 |
feat(facets): all-facets panel with under-filter counts (faceted plan Phase 7)
The 9 Batch A facets could not be SET from the web UI, the facets
endpoint had no in-app consumer, and the registry's libraryTypes hints
rendered nowhere. This phase closes all three.
- ONE UNION-ALL sqlc query (FacetCountsUnderFilters) computes all 14
listable facets in a single round trip: each arm omits its OWN
facet's filter block and carries every other registered facet's
block (copied mechanically from SearchMediaItemsUnified) — classic
faceted semantics: with filters active each facet counts under the
others; with none, blocks are neutral and the counts are the v1
global-per-library ones. One query serves both cases. The 14
per-column Batch C queries are deleted.
- Endpoint: no facet param → the new all-facets shape
({facets: {param: [{value, display?, count}]}}), filters bound via
the shared ParseMediaSearchParams, library_id optional (the
All-Libraries view, family lens still enforced); facet=X keeps the
pinned single-facet shape (now routed through the same query with
neutral filters).
- SSR partial GET /media-items/facet-panel: templ renders the drawer's
value lists — registry order, Batch D library-type scoping
(FacetsForLibraryType: manga_type on manga, reading_direction on
manga+comics, narrator on audiobooks; All view = union), prettified
genre labels + language display names, active values marked. Family
lens: out-of-set explicit library_id is 403.
- Drawer wiring: the panel fetches on drawer open and after every
apply with the CURRENT filter query string; a value click toggles
that facet's comma-joined filter (hidden input where the drawer has
no field — same as deep links) and re-applies the form; chips and
URL stay in sync. Values are server-rendered (rendering doctrine);
bookshelf.ts owns only the click/fetch orchestration.
- prettifyGenre moves to internal/utils as the canonical
implementation (services' display enrichment shares it; templ wraps
it; the TS port in bookshelf.ts is unchanged).
- Drift guard extended: FacetCountsUnderFiltersParams must carry every
registry facet's param, the bools as pgtype.Bool.
Tests: under-filter exactness (own-block omission, OR-within,
AND-across), family lens (hidden values absent; 403 on both shapes),
all-shape + pinned-shape coexistence, partial rendering with Batch D
scoping + active marking + 403. Full container suite green except the
documented pre-existing TestAdminCreateUser red.
|
||
|
John O'Keefe
|
6152137156 |
fix(tests): repair stale pins surfaced by the Phase 6 full-suite run
Two suite failures, neither caused by scanner code (the Phase 6 diff
touches no handler):
- search_test invalid_library_id expected the pre-family-lens 404;
|
||
|
John O'Keefe
|
2c5ad014c0 |
feat(scanner): multifile folder/zip sidecar detection (faceted plan Phase 6)
Reverses the Phase 4 scope cut: for EXISTING folder/zip audiobooks a
sidecar edit was invisible to everything, including force rescan —
updateAudioBookItem writes only scan-derived fields, and these formats
often have no useful embedded tags, so the sidecar is the only source.
- Change fingerprint gains the manifest compare in both processors:
changed := size differs || track payload differs || manifest drifted.
The ladder runs against the bulk-loaded map entry (stat pairs only,
zero extra queries); NULL stored manifest backfills without
re-extraction (deploy-day rule, same as single-file).
- Manifest-changed applies the once-extracted book metadata to the
editorial columns (updateMediaItemWithMetadata, overrides protected)
and re-runs the playlist hook. The apply runs BEFORE the focused
payload write: the generic update's format-group pass derives mime
from the sentinel (none) and chapters from metadata.ChapterCount
(unset) — the payload write must win (rescanMultiFileAudioBook's
ordering). Track-only changes keep the existing editorial-keep
update; force rescan keeps it too.
- Manifests seeded at multifile create (folder names derive from the
sentinel — the exact representative the extractor reads — via
detectSidecarChangeWithRepresentative, so manifest and extractor can
never disagree; a directory-basename cover is ignored by the
extractor and therefore not tracked).
- Zip parity fixes: the fast path now lifts archive state like the
folder path always did (a returned zip book stayed archived
forever), and the change branch handles the unchanged-archived case.
- Single-file sidecar change path: extract-failure guard — the old
non-nil empty &MediaMetadata{} fallback bypassed the update path's
filename-title guard (preExtracted == nil only) and would blank the
title on a failed extraction; the row is now left untouched and the
manifest refreshed. Empty-success extraction (sidecar removed from a
tag-less book) now passes nil so the create-equivalent chain (tags →
filename) re-derives instead of blanking.
- Multifile changes now count in the scan summary (changedItems);
they were previously uncounted.
Tests: sentinel-name derivation unit pin; folder sidecar edit
(title + collection sync + fresh manifest + steady-state skip),
track-only change keeps editorial, zip sidecar edit + archive
restore, sidecar removal falls back to filename (the wipe-guard's
testable sibling). Full container suite: green except the documented
pre-existing TestAdminCreateUser red.
|
||
|
John O'Keefe
|
5245f292ca |
feat(scanner): quiet scans, enqueue coalescing, watcher probe, sibling cache
Part 2 of the faceted plan, Phase 5 (steps 5-6): the scan that used to print ~4 lines per unchanged file now prints ONE summary line, and a poll tick landing mid-scan no longer stacks a second full scan. - Quiet scans (§4): the walk loop is silent — inserts, changes, and errors log inside processMediaFile; the duplicate 'Successfully processed' branch (both branches printed it) and the per-file 'processing / already exists / same size, skipping' chatter are gone. Summary carries the trigger and a changed counter: 'Scan of <folders> (trigger: poll): N files, N changed, N new, Xs, N errors'. Triggers threaded through every enqueue site: poll (startBackupScan), watcher (dirty-dirs flush), admin (scan API), initial (startup scan). - Enqueue coalescing (§5): Worker.HasActiveScanForLibrary (JobResult gains additive job_type so only SCAN jobs coalesce — an import sharing the library id never blocks one). enqueueLibraryScan skips silently; the admin scan API answers 409 with the active job id. Enforces the one-scan-per-library invariant the bulk scan-state map leans on. - Watcher capability probe (§4): ONE Add on a root folder; failure disables the watcher with a single startup line (replaces the per-directory warning spam on inotify-broken deployments). The watcher stays an optional latency reducer over scheduled scans. - Per-directory sibling cache (§6): hasSiblingBookFileCached caches the ReadDir per directory for one scan — loose-image manga re-stats thousands of images sharing a handful of directories. Tests: TestHasActiveScanForLibrary (unit — running/pending coalesce, terminal jobs don't, non-scan jobs don't, empty library never, nil worker safe). Full scanner suite green in the container. |
||
|
John O'Keefe
|
6750e9e0dc |
feat(scanner): manifest-gated sidecar change detection + playlist sync on change path
Part 2 of the faceted plan, Phase 4 (steps 3-4). Size-only detection
was blind to metadata-sidecar edits; a size-stable book is changed by
EDITING its metadata.opf/metadata.json/cover — previously invisible
until a force rescan.
- Sidecar manifest fingerprints (media_scanner_manifest.go): the
manifest stats exactly the names the extractors consume (metadata.opf,
metadata.json, findSidecarCover patterns incl. <basename>.<ext>), so
manifest and extractors can never disagree. Shape v1:
{v, entries:[{n, s, m (unix nanos — second granularity would hide
edits landing in the backfill scan's own second), h (sha256 — covers
included: the repoint ladder needs a baseline hash for any stat-
drifted sidecar; a hashless cover turns harmless mtime drift into a
re-extraction, the exact flood this design kills)]}.
- Evaluation ladder in cost order: stat pairs identical → skip with
ZERO bytes read; stat drift → hash that sidecar (equal = host repoint
reconciliation, stat pair refreshed, no re-extraction; differing =
changed); NULL stored manifest → backfill only, NO re-extraction
(deploy-day rule: the first scan after release must not look like an
mtime flood).
- Change path: re-read full row, extractMetadata once and pass it
through updateMediaItemWithMetadata (preExtracted — metadata_overrides
stay protected), run syncPlaylistCollectionsAfterCreate (Correction 2:
a reading-order entry added to an EXISTING book now creates its
collection; add-if-missing is idempotent), store a fresh full
manifest. Creates seed their manifest at insert time.
- Multifile folder/zip books intentionally keep Phase 3 behavior: their
change path does not consume sidecar extractors, so gating them would
flip books 'changed' without picking anything up (plan correction —
the plan's §3 multifile note assumed otherwise).
- UpsertMediaScanState + GetMediaScanState queries.
Tests: media_scanner_manifest_test.go (ladder unit tests — fast path,
repoint, same-size drift, appear/vanish, unhashable legacy entries);
TestScanManifestLifecycle (seeding, unchanged-skip stability, repoint
reconciliation with stat-pair refresh assertion, same-size edit
consumed, playlist sync on the change path + idempotency, no-leak
guard). Container runner.
|
||
|
John O'Keefe
|
693053a9f5 |
perf(scanner): bulk scan-state load — one query per library scan
Part 2 of the faceted plan, Phase 3 (steps 1-2). The periodic scan
issued one GetMediaItemByFilePath per scannable file (~3,094 queries
per scan in prod, nearly all answering 'unchanged, skip') plus one
GetLibraryByFolder per file for folder resolution.
- NEW media_scan_state table: the scanner's per-item fingerprint
cache. Deliberately NOT a media_items column — the library-scoped
list emits raw sqlc rows as JSON (
|
||
|
John O'Keefe
|
bbedbfb13f |
feat(facets+ui): clickable facet values everywhere, genre prettify, active-filter chips
Frontend half of the faceted-navigation Phase 2. 'Everything filterable should be clickable': every displayed facet value links to the bookshelf pre-filtered by its raw stored value; display forms are prettified at render, never in the DB. - templates/facet_links.templ: FacetLink (inline links) + FacetBadge (badge links) to /bookshelf?<facet>_filter=<value>; the raw value rides the param while the shown text is prettified, so matching and display can never disagree. Series/tags keep their dedicated pages by design; the card media-type icon stays unlinked (a derived kind, not a column value — deep-linking it would send wrong values). - book_detail.templ: publisher, language, edition, genre, imprint cells + narrators line + age-rating/story-arc badges are now links. - prettifyGenre (templates/utils.go + unit tests): title-cases only entirely-lowercase values; small words stay down except first position; & preserved. TS port for chip labels documents the twin. - bookshelf.templ + bookshelf.ts: active-filter chips bar — Alpine renders chips from client-local form state (sanctioned island); one bubbled htmx:afterRequest hook keeps chips + shareable URL in sync; removeFilterChip clears the facet and re-applies. URL deep links now accept all 18 facet params (hidden inputs where the drawer has no field). Pins changelog: Batch B/C entry in docs/contract_pins.md. Rendering ownership guard (make lint-rendering) green. |
||
|
John O'Keefe
|
7e586df48a |
feat(facets): facet registry, Batch A server params, genre model v2, facets endpoint
Phase 1 + backend of Phase 2 of the faceted-navigation plan. One
declarative facet registry (services/facet_registry.go) drives the
bookshelf filter system; a drift-guard unit test reflects over the
generated SearchMediaItemsUnifiedParams so a registry entry and its
sqlc.narg block can never disagree silently.
- Nine new filter params on SearchMediaItemsUnified: story_arc,
imprint, edition (scalar-fuzzy), age_rating, format_group,
manga_type, reading_direction (exact), explicit, abridged
(presence-bound bools). Comma = OR-within multi-select on every
text facet; AND-across-facets unchanged.
- Genre model v2 (user rule: a subgenre is not a genre, it's a tag):
genre_lexicon.go rebuilt as shelf set + subgenre->shelf map;
pickSmartGenre picks direct shelf > mapped subgenre > Fiction floor
> empty (unknown-only stays tags-only). Type-words (Manga, Comics,
Graphic Novel) join the demote list. Both 'adventure' and 'action &
adventure' are shelves — provider data uses both and the verified
traces pin them.
- Language filter UX: comma elements normalize through the ingest
language map, so display names ('English') match rows stored as ISO
codes ('en'). The language autocomplete now serves display names.
- NEW GET /api/media-items/facets?facet=X&library_id=... — registry-
driven value+count listing (14 static per-column queries, family-lens
join); language values carry display names; bool/range facets 400.
- Family-lens gates: an explicit hidden-library library_id is now a
403 on the search endpoints (was a silent empty set) — JSON branch
and HTMX grid partial both.
- handlers.ParseMediaSearchParams is the SINGLE param parser for the
search JSON endpoint and the HTMX grid partial; the HTML branch
previously dropped publisher_filter and narrator_filter silently.
Pins: docs/contract_pins.md changelog entries (additive request
params; new facets endpoint). Tests: facet_registry_test.go (shape +
SQL drift guard + library-type union), series_parsing_test.go genre-v2
cases, facets_integration_test.go (param filtering e2e, counts,
language display, 403s) — container runner.
|
||
|
John O'Keefe
|
7537063cb7 |
feat(filters+nav): author navigation end-to-end — arrays on every media response, deep-linkable bookshelf filters, clickable authors
Completes the A2c acceptance scenario: click an author on a book detail page -> land on the bookshelf filtered to everything that author wrote (including their share of co-authored books). - Media detail + unscoped list resolved maps expose subtitle, explicit, authors, publishers (the library-scoped raw branch and the search endpoint already did — raw rows follow the SELECT). - SearchMediaItemsUnified SELECT carries the new columns; the search endpoint serializes the row directly, so clients get them there too. - book_detail.templ renders each author as a link to /bookshelf?author_filter=<name>. - Bookshelf honors filter params in the URL on load (author_filter, series_filter, genre_filter, publisher_filter, tags_filter, narrator_filter, language_filter): pre-fills the form, applies it to the first grid load. Comma-joined values are multi-select via the server-side string_to_array split. Filtered views are shareable. Pins widened: search rows (+7 keys), resolved keys (+4); changelog entries in docs/contract_pins.md. Android follow-up plan written to ~/.opencode/plan/bookhoard-app-v2-compat.md. |
||
|
John O'Keefe
|
f94e7472b4 |
feat(filters+ui): per-element author/publisher filters, playlist toggles, Up Next API, reorder UI, dashboard badges
Phases 4 + 5 of the metadata overhaul (plan §4, §5) plus §3 handler wiring. Filters (§4.1): author_filter/publisher_filter now match per-element against the authors[]/publishers[] arrays — a co-authored book matches any one of its authors — with comma-joined params acting as OR-within-facet multi-select (string_to_array + cross unnest). Legacy single-value params unchanged on the wire; series/genre/language filters stay on their varchar columns; narrator filter already per-element. Collections API (§3.3-3.5): collection detail carries viewer + hide_finished into the listing query and exposes up_next (visible- library-enforced reading-queue head); PUT /api/collections/:id/items/order (owner-only, full-order rewrite); UpdateCollectionRequest gains hide_finished/show_up_next ([] string last-value-wins for the hidden-false-sibling checkbox pattern) merged server-side OVER stored view_settings so imported_from_series provenance survives edits; EditCollectionModal renders the toggles from view_settings. Dashboard (§5.5): sections JSON carries is_playlist/complete (dashboard.ts is the AGENTS-documented JSON-render exception; first paint stays templ). Fully-read playlists render '✓ Playlist complete' instead of an empty rail; imported playlists get a 📖 Playlist badge. Frontend: book detail shows subtitle + full authors + Explicit badge; edit modal gains subtitle, comma-separated authors/publishers (client- split to arrays exactly like narrators), step=any series number, explicit checkbox; collection book cards get playlist move ↑/↓ buttons persisting the full order via the reorder endpoint (templ partial re-render on failure — DOM order is client-local, API order is truth). AGENTS.md gotcha #3 amended per plan §3.7: user collections are dashboard sections by default; the show_on_dashboard premise is retired. Verified: full affected integration sweep green (pins widened for up_next/is_playlist/complete — changelog entries in docs/contract_pins.md); unit suite green; go build + templ + vite green. |
||
|
John O'Keefe
|
1dbb0ce406 |
feat(collections): playlist auto-creation, position ordering, hide_finished, Up Next, dashboard eligibility
Phase 2b + 3 of the metadata overhaul (plan §2.4, §3.1-3.7). Auto-playlist collections (§2.4): the scanner turns obviously-playlist sidecar series (explicit '(Reading Order)'/chronological/publication/ in-order/timeline markers only — precision over recall) into curated static collections: upsert by view_settings.imported_from_series then name (rename-safe), position set ONLY on insert (clean integers; 1.5/1a stay unpositioned rather than guessing), membership add-if-missing, never removed or re-ordered by rescans, provenance in description + view_settings, show_on_dashboard true (dashboard-eligible by design), 'scanner.auto_playlist_collections' setting (default on, Settings Registry + schema seed). Ordering (§3.1-3.2): collection listings order position ASC NULLS LAST then added_at ASC (was added_at DESC — walked playlists backwards); PUT /api/collections/:id/items/order applies full-order rewrites (owner-only, service-gated). hide_finished + Up Next (§3.3-3.4): GetCollectionItemsForDashboard gains viewer_user_id + hide_finished (LEFT JOIN reading_progress, COALESCE percentage < 0.999) and viewer_percentage rides the row for card graying; GetCollectionUpNext serves the collection's reading-queue head with the family-lens visible-library array enforced. Dashboard (§3.5): user collections are dashboard-ELIGIBLE BY DEFAULT — the show_on_dashboard gate (never writable for user rows since the column shipped) is gone from GetUserCollectionsForDashboard; hiding stays dashboard-prefs (hidden_collections) territory; hide_finished filtered-everything now emits a Complete section instead of silently vanishing; sections JSON carries is_playlist/complete. Series page ordering hardening (§3.6): GetSeriesBooks tiebreaks by title ASC — NULL/tied numbers previously returned heap (scan) order, which is what the user saw. Frontend custom-section-builder note: the builder has sent show_on_dashboard: true since its creation; the API never bound it. User-row visibility is now intentionally prefs-governed, so the field stays unbound by design (documented in code). Pins widened (docs/contract_pins.md): collection detail + up_next; dashboard sections + is_playlist/complete; new reorder endpoint recorded. Verified: TestPlaylistCollectionsEndToEnd (new, full e2e: sidecar series split, collection auto-creation with position 7, rescan idempotency, owner-only reorder, finished-filter + completion marker + Up Next drain) plus the full affected integration sweep green. |
||
|
John O'Keefe
|
7e4c97f639 |
feat(scanner): fix ABS sidecar series parsing; three-way series mapping; smart genre; language codes; title-derived series
Phase 2a of the metadata overhaul (plan §2.1-2.3, §2.5-2.9). The headline bug: ABS writes sidecar series as an array of STRINGS (["The Witcher #3", ...]); the parser only accepted {series, sequence} OBJECT arrays, so every ABS-matched book silently lost its series (found live: 9 sidecars, zero series ingested; only folder-structure Malazan survived). parseSeriesCandidates now normalizes string arrays, object arrays and Libation plain strings. Three-way series assignment (plan §2.2): - primary bibliographic series -> series/series_number columns - additional real series (Drizzt: Icewind Dale #1 = Legend of Drizzt #4) -> alternate_info JSONB, comic-established shape - ordering constructs ('The Witcher (Reading Order)') -> PlaylistSeries candidates; NEVER the series column. Detection is explicit-marker-only ((reading order|chronological|publication order|in order|timeline) as parenthetical/suffix) — precision over recall; a series merely containing 'timeline' mid-name stays a series. Sequence policy: leading-number extraction ('Book 2' -> 2, '2, Dramatized Adaptation' -> 2, '1a' -> 1); lossy extractions preserve the raw text in alternate_info.series_sequence_raw. Smart genre (deterministic lexicon): compound Audible blobs split ('Epic, Action & Adventure, Military, Sword & Sorcery' -> 4 candidates), alias-normalized, format noise demoted ('Audiobook' never wins), most-specific canonical genre takes the genre column, all candidates stay in tags. Language: display names normalize to ISO 639-1 on ingest; ROOT-CAUSED the update-path drop — CreateMediaItem call site never set Language (explicit NULL over the column default), the update path did; create now carries it. Title-derived series fallback (§2.7): 'Baptism of Fire: The Witcher, Book 3' 2-level folders split conservatively (only when the post-colon tail matches ', Book N'), subtitle parsed for series+number, blanks-only precedence, matching-name number fill, zero display mutation. Works with no sidecar and no network. Ingest fixes: subtitle/explicit land in their columns (audio tag subtitle stops joining into title); full authors[] array (was [0]- only); sidecar descriptions sanitized via bluemonday UGC policy (vendored). metadata_overrides: verified scan-update path already applies ApplyMetadataOverrides; subtitle/explicit/authors covered by the Phase 1 override adaptation. Tests: new series_parsing_test.go (string-array sidecar e2e, Drizzt alternate_info, lossy raw preservation, playlist classification, title-derived fallback incl. no-mutation + blanks-only, genre lexicon, language map); audio-tag subtitle pin updated to column behavior; calibre/ABS 2.5-series_index pins widened per float change (changelog'd in phase 1). Deferred to 2b: auto-created playlist collections (§2.4) — settings toggle, collection upsert, ownership. |
||
|
John O'Keefe
|
646477afad |
feat(schema): subtitle/explicit columns, authors+publishers arrays, float series numbers
Phase 1 of the series/collections/metadata overhaul
(~/.opencode/plan/series-collections-metadata-overhaul.md).
Schema (in place, idempotent):
- media_items.series_number INTEGER -> DOUBLE PRECISION: real-world
ordinals are decimals (0.5, 1.5 novellas); INTEGER silently truncated
them (calibre 2.5 pinned as 2 in tests).
- + subtitle TEXT, explicit BOOLEAN (sidecar ingest; ABS explicit /
Audible isAdult).
- + authors/authors_search/publishers/publishers_search TEXT[] NOT NULL
(narrators A2a precedent). Arrays are write truth; author/publisher
varchars become trigger-regenerated display strings
(sync_author_publisher_display) so every varchar consumer (trgm
search, facets, API, OPDS/Kobo) keeps working unchanged.
- Row-level NULL normalization for the new arrays in
set_library_type_name (A2 lesson: Go nil -> explicit NULL -> 23502).
- Backfills ARRAY[author]/ARRAY[publisher] — deliberately no
comma-splitting ('Tolkien, J.R.R.' would corrupt; scanner kept only
authors[0] historically).
- collection_items.position INTEGER + (collection_id, position) index
for playlist ordering.
Go: sqlc regen; MediaMetadata/handler params gain Subtitle/Explicit/
Authors/Publishers (legacy singles still accepted, arrays win);
SeriesNumber int32 -> float64 end to end; metadata_overrides adapted
with legacy override keys ('author'/'publisher') preserved so existing
user overrides survive; bulk author edit writes the array.
Pins widened (docs/contract_pins.md): raw sqlc row key set + the six
new columns; changelog entry added.
Verified: unit suite green; integration sweep (bulk/smart collections/
scanner/stats/sync/queue/calibre/pins/audiobooks) green against the
shared dev DB after in-place migration. Pre-existing unrelated red:
TestOwnershipAndAdminTestSuite/TestAdminCreateUser timezone inheritance
fails identically at the pre-change commit (live shared state).
|
||
|
John O'Keefe
|
f7dc8189a1 |
feat(css): cross-document view transitions
Release / build-and-push (push) Successful in 3m15s
@view-transition { navigation: auto; } lets Chrome 126+ / Safari 18+
crossfade same-origin navigations — sidebar links and the series
page's library-switch navigation get a subtle fade with zero
JavaScript, softening the switch to full SSR navigation. Pure
progressive enhancement: Firefox and older browsers simply navigate
instantly. style.css regenerated via build:css:prod.
v1.17.1
|
||
|
John O'Keefe
|
a8641ab343 |
chore(ui): bump the main.js cache-bust stamp to v6
web/static/*.js is gitignored and built at image-build time, so the hardcoded ?v= on the module script tag is the ONLY mechanism that evicts a stale bundle from browsers. Shipping the series twin deletion without a bump would let cached main.js (with the deleted renderer) keep running on prod, making the paginator fix appear not to work. Found while shipping; easy to miss because the bundle never appears in git status. |
||
|
John O'Keefe
|
1ce66b2213 |
chore(templ): warn against change handlers on the bookshelf library select
The bookshelf select (#library-select) sits inside the htmx filter form and the SSR route honors ?offset= — wiring a change handler or hx-trigger="change" on it would let header.ts's synthetic restore event re-fetch the grid at offset 0, the same stomp class that broke the /series paginator. Comment-only guard; regenerated bookshelf_templ.go carries it into the emitted markup. |
||
|
John O'Keefe
|
18f52d240b |
docs(agents): record the dashboard sections rendering exception
renderDashboardCollections/renderSectionHTML re-render dashboard sections from /api/dashboard/sections JSON on library switch, settings save, and refresh — deliberately kept in TypeScript to preserve the switch fade on the app's most interactive page. First-paint section markup remains templ (Dashboard handler in internal/router/frontend.go). Recorded in the Rendering ownership section as a sanctioned exception so it is not silently 'fixed' without restoring a templ-partial fade path, and so the pattern is not cloned to new pages. (User-directed: the dashboard keeps its fade; every other page stays templ-rendered.) |
||
|
John O'Keefe
|
220d97cfe0 |
test(series): pin the SSR pagination contract under tied timestamps
New integration test mirroring a prod bulk import: a dedicated test library with 30 series whose items share ONE identical created_at (single-transaction NOW()), the exact data shape that made prod pagination look broken. Pins, via the real SSR /series route: - page 1 renders exactly 24 deterministic series (01-24) with chip "Page 1 of 2", no Previous, and a Next href carrying library_id and page (templ escapes & as & inside attributes) - page 2 renders the remaining 6, disjoint from page 1, chip "Page 2 of 2", no Next, Previous href correct - missing page param defaults to page 1; page=0/-3/two clamp to page 1 Guards both the SQL tiebreaker and the /series handler contract; would fail on any return of the stomp or an unstable sort. |
||
|
John O'Keefe
|
38efbff7e1 |
fix(db): total ordering for series OFFSET pagination
GetDistinctSeries ordered by MAX(created_at) DESC alone. Bulk imports inserted in a single transaction share one NOW() timestamp, leaving large numbers of (library_id, series) groups tied on MAX(created_at); Postgres does not guarantee tie order between executions, so LIMIT/OFFSET pages could repeat or drop series across page loads. Add the GROUP BY keys (library_id, series) as tiebreakers so the sort is total and deterministic. Comment documents why the tiebreaker is required so it is not dropped as noise. sqlc regenerated (interface unchanged; the comment flows into querier.go docs). |
||
|
John O'Keefe
|
f4414aa5b6 |
style(series): plain-text page indicator instead of the chip pill
"Page N of M" now renders as small secondary-color text on the page background rather than an accent chip (user preference; the old TS twin happened to render this way and the chip was only being seen after JS took over). With the twin deleted, this templ markup is the only paginator rendering, so the look is stable. |
||
|
John O'Keefe
|
3daa834a94 |
refactor(series): drop the TS rendering twin; library switch navigates SSR
web/src/series.ts carried a full TypeScript twin of the templ page — cards and paginator rendered via innerHTML from /api/series JSON, always hardcoded to page 1. Combined with the header restore dispatch (previous commit) it made every /series load stomp server-rendered pagination back to page 1, which is the paginator bug reported on prod (9 pages of series, Next kept landing on page 1, per-library only). Rendering ownership (AGENTS.md): this page now has NO client renderer. - grid and paginator are rendered only by templates/series.templ - a library switch is a plain SSR navigation to /series?library_id=...&page=1, keeping the URL the single source of truth for both library and page state - drops the duplicate /api/series fetch that ran on every page load rendering-allowlist.txt: series.ts baseline entry removed — the file no longer builds HTML anywhere. |
||
|
John O'Keefe
|
202a090811 |
fix(ui): library restore no longer re-fires change on synced dropdowns
restoreLibrarySelection() ran on every page load via the header's x-init and always dispatched a synthetic `change` on #library-select after setting its value. On SSR pages the dropdown already renders the stored library (setSelectedLibrary keeps the cookie in sync and resolveLibrary renders from it), so the synthetic event re-ran the page's switch handler, whose fetches are hardcoded to page 1 / offset 0 — stomping freshly rendered content. Visible impact: the /series paginator snapped back to page 1 on every navigation whenever a specific library was stored (All Libraries short-circuits restore, which is why it only failed per-library). Silent impact: dashboard and collection-detail pages re-fetched their content on every single load. The restore now re-fires `change` only when the dropdown genuinely drifted from storage; the common synced case is a visual no-op. |
||
|
John O'Keefe
|
ee98c0fb3b |
docs(dev): test-running policy + media-fixture rules; ignore testdata
Release / build-and-push (push) Successful in 3m29s
Integration tests require the containerized dev DB and bind-mounted paths — a bare host `go test ./cmd/server/tests` fails on database auth on every machine not running the compose stack (the old doc prescribed exactly that flow). development.md now documents the container runner and the iteration rule: run ONLY your own tests via a filtered container invocation while working, and reserve full-suite runs for pre-commit validation — several tests read live shared state (system_settings, real library folders, the periodic backup scan's websocket broadcasts) and flap on data or timing a change never touched. Binary media fixtures are never committed: internal/services/ testdata/ is gitignored by policy (the audio metadata tests read real-format binaries that ffmpeg cannot synthesize — Audible freeform atoms — so they live untracked on the machines that run the suite). The new "Media fixtures" section carries the ffmpeg recipes for every supported container, tag writing, embedded covers and ffmetadata chapters, plus the manual end-to-end recipe for the live system (one book per subfolder — a directory with ≥2 audio files ingests as one folder book).v1.17.0 |
||
|
John O'Keefe
|
79ce5ab89d |
feat(web): multi-track audiobook player — per-track src switching, global-seconds position
The audio reader pointed <audio src> at ONE file — correct for m4b but impossible for folder-of-tracks and zip books (the element holds a single source). The player now generalizes to a track list built from the /chapters manifest: single-file books are exactly one track whose url is the whole file (zero behavior change — same element, same URL flow), multi-file books swap src at track boundaries. Mapping helpers (exported, unit-testable): trackIndexFor resolves a global position to the owning track, localOffsetFor/globalSecondsFor convert between global seconds — the coordinate system of chapters, bookmarks and audio_offset_seconds — and (track, local offset). Every consumer keeps global semantics: scrubber, skip ±N, bookmark saves, ?loc=audio: deep links and progress pushes are byte-identical payloads server-side; the server contract does not move. Mechanics: setActiveTrack swaps src and queues the position for loadedmetadata (setting currentTime before the new metadata loads resets it); ended advances to the next track and keeps playing; timeupdate reports globalPosition(); playbackRate/volume re-apply after swaps; src is deferred until the manifest loads (data-file-url is a directory/zip root for multi-file books, not playable) with the catch branch falling back to it for offline single-file starts. |
||
|
John O'Keefe
|
6216bd9bcf |
feat(audiobooks): multi-file books (folder-of-tracks + zip) + exhaustive metadata ingestion
Multi-file book ingest: a directory of ≥2 audio files (recursively —
disc-part folders like CD1/Disc 2 merge upward, siblings never merge)
ingests as ONE book whose file_path is the book-root directory; a zip
of audio tracks is its own book (file_path = the archive). Both write
the documented chapters abstraction {index,title,start,end} into
audio_metadata and add additive keys: layout ("folder"|"zip") and a
tracks[] manifest (per-track path, global-seconds start/end, duration,
size). Pre-grouping one-track strays under a book root are absorbed on
rescan (annotations cascade). Rescans take a stat-only fast path
(track count + total size + known durations unchanged → no re-probe —
a 1189-track book must not re-ffprobe every auto-scan); a zero-duration
book counts as changed so a ffprobe-equipped rescan fills durations in.
Folder-book roots count as present in the archive lifecycle pass, and
per-item rescan (RescanMediaItem) routes multi-file books through a
book-level refresh — the generic updater would read the root/zip as if
it were the book file and clobber format_group with "unknown".
Serving: GET /api/media-items/:id/chapters gains additive layout +
tracks keys (urls → the new endpoint). New GET
/api/media-items/:id/audio/:trackIndex streams one track — folder
tracks served from the real file (full Range via ServeContent), zip
tracks via a reopen-and-skip ReadSeeker over the entry (deflate has no
random access; acceptable at audio bitrates; stored entries stay
section readers). Both endpoints gate on library visibility (403 on
hidden libraries — family-lens rule). ServeFile and OPDS DownloadBook
refuse directory paths (folder books have no single-file download).
Web player (next commit) consumes the manifest; progress/bookmarks/
deep-links keep their global-seconds coordinates, so the server-side
progress model is untouched. sync learns FormatGroupAudiobook
(percentage passthrough in ConvertProgress); detectFormatType stops
reporting "unknown" for audio.
Metadata ingestion (Batch B) — read everything available, degrade
gracefully, documented precedence sidecars → folder structure →
embedded tags → filename: ffprobe tags are now a case-insensitive
lookup map (freeform keys pass through verbatim); composer fills
narrators (Audible's own convention, kept as ONE string — no reliable
multi-narrator delimiter); grouping + Audible freeform atoms
(SERIES/SERIES-PART/SUBTITLE/ASIN/ABRIDGED) fill series/index/
subtitle-join/ASIN/abridged (explicit sidecar abridged-false is
guarded); date/genre fall back to tags; album/album_artist/artist
fill title/author. Genre persists at create for all formats.
Embedded cover art (mp3 APIC, m4b covr, flac picture) extracts via one
optional ffmpeg call as a "<book>.cover.jpg" sibling — sidecar covers
win. CUE sheets (FILE-matched, INDEX 01 at 75fps) are the lowest-
priority chapter source under sidecars and embedded m4b chapters. A
lone m3u/m3u8 in a book root imposes playlist ordering + #EXTINF
names; (disc, track) tags reorder tracks when coverage is complete.
extractMetadata's default branch no longer pre-fills the filename as
title (tags must outrank it) — final filename fallbacks now exist on
BOTH the create and rescan paths (TestRescanKeepsFilenameTitle pins
that rescans cannot blank a title); tagless zip books take the zip's
own name, not their parent folder's.
Accepted containers widen to .m4b .m4a .mp3 .flac .zip .opus .ogg .oga
.aac .wav with real media types registered (audio/ogg per RFC 7845 for
Opus-in-Ogg) — all browser-playable; .wma deliberately excluded.
Tests: unit tests for CUE parsing, playlist hints, tag fallbacks
(synthetic ffprobe reports — ffmpeg cannot write Audible freeform
atoms), ordinal parsing, tag ordering, natural sort, grouping rules;
integration suite (11 tests) over the live dev DB: folder/zip ingest +
idempotency, stray absorption, chapters manifest, per-track streaming
with Range, hidden-library 403s, embedded covers + sidecar-wins, CUE
chapters, embedded m4b chapters, playlist ordering, tag fallbacks, all
five new containers with content-type assertions, rescan title pin.
Fixture media is real committed-format binary content living untracked
under internal/services/testdata/ (gitignored by policy; recipes in
docs/contributing/development.md → Media fixtures). Test image now
ships ffmpeg to match the runtime's audio toolchain.
|
||
|
John O'Keefe
|
24228e7147 |
test(scan): cancellation endpoint semantics and mid-run safety
- endpoint semantics: non-admin cancel is 403, unknown job is 404 - mid-scan safety: start a scan of 1200 unparseable .epub files and cancel it, then pin the invariant that no media item is marked missing or archived. The canary item's file is lexically LAST in the folder, so an abort that (incorrectly) ran the archive pass would treat it as unvisited and trip the assertion. Assertions hold under every interleaving - whether the cancel lands mid-walk or the scan already finished - so the test cannot flap against timing. - re-cancelling a terminal job is a 409 The test library is created through the shared createTestLibraryWithFolder helper, which registers API-delete cleanup so runs leave no residue in the dev database. Worker-level unit tests live in internal/services/worker_cancel_test.go (previous commit). |
||
|
John O'Keefe
|
e1153b2c4b |
feat(admin): cancel buttons for in-flight scans + cancelled state UI
- per-row Cancel button in the scan-progress card; listeners are bound after the TS island renders its rows (module scope has no inline handlers), and the button is removed once its job turns terminal - cancel-all icon button in the card header: resolves the server's active job list first, so it also covers scans this page never started (e.g. started before a refresh), then cancels each one - polling treats cancelled as terminal, averages overall progress only over jobs that can still advance (so the bar can reach 100%), and the results panel reports cancelled scans instead of claiming "Scan Complete!" - sidebar spinner resets on the scan_cancelled websocket event (same quiet handling as scan_error) - setup wizard shows "Cancelled" as a terminal status in its poller No new innerHTML assignment sites - rendering-ownership baseline unchanged. |
||
|
John O'Keefe
|
5b81e19f2f |
feat(scanner): POST /api/scanner/cancel/:jobId (admin)
Cancels a pending or running scan job. Responses: 200 with the job id and status on acceptance, 404 for unknown jobs, 409 for jobs already in a terminal state (completed/failed/cancelled). Additive route in the existing admin scanner group; no contract pins affected. |
||
|
John O'Keefe
|
30ecdd4bf3 |
feat(scanner): ScanFolders honors its context; cancelling never archives
ScanFolders never checked its context, so a cancelled job's walk ran to completion no matter what the worker's cancel did. - ctx checks at every stage boundary (file-counting pass, per folder, before and during the archive pass) and inside the walk callback (fs.SkipAll): the file currently being processed finishes, then the scan aborts with context.Canceled, which processJob classifies as a cancelled job rather than a failed one - the archive/purge pass is skipped entirely when the scan was cancelled. It only visits what the walk visited, so after a partial walk the unvisited (but on-disk) files would be treated as missing, archived, and eventually purged. Cancelling is a UI convenience and must never destroy library data. |
||
|
John O'Keefe
|
b2220756e2 |
feat(worker): real per-job cancellation for the background job pool
CancelJob existed but was dead code: it only flipped a status in the in-memory results map - it never cancelled anything, and processJob's unconditional completion write overwrote the marker anyway. - derive a per-job context in processJob (parent is the handler ctx for manual scans, Background for auto-scans) and register its cancel func under the job id; previously every manual scan shared the handler's process-lifetime context, so no targeted cancel was possible without killing everything - skip execution at dequeue for jobs cancelled while still queued - record a pending JobResult on BOTH enqueue paths (Enqueue used by auto-scans, EnqueueJob used by the API): queued jobs were invisible to status polling (404 until a worker picked them up) and could not be cancelled before execution started - cancelled jobs keep their last real progress instead of a fake 100% and drop the meaningless "context canceled" error text - broadcast the new scan_cancelled websocket event for cancelled scan jobs instead of scan_complete, so clients reset their scanning indicators rather than reading the abort as a finished scan - sentinel errors (ErrJobNotFound, ErrJobNotCancellable) so HTTP handlers map outcomes to 404/409 without string matching Unit tests (worker_cancel_test.go) pin: queued-cancel never executes, running-cancel invokes the context, error mapping, pending recording on both enqueue paths. |
||
|
John O'Keefe
|
d74db03cf3 |
feat(sync): unlinked-books queue — ingest, page route, dashboard banner
Release / build-and-push (push) Successful in 3m7s
The identification queue for device files the sync cannot match is now live end-to-end. Previously the whole feature was dark: unmatched device files were silently dropped (losing their reading progress), the queue table sat empty by construction, and the page template had no route. - Ingest: the KOReader sync loop, on an unresolvable file, upserts unlinked_books (one row per device + content id; re-syncs bump last_seen_at and refresh device-reported metadata) instead of dropping the file. Schema gains the (device_id, content_id) unique index the upsert needs. - Page: GET /unlinked-books routes the existing page template, listing the caller's own unresolved rows (ListUnresolvedUnlinkedBooksForUser — devices.user_id scoping; no cross-user leakage). - Dashboard banner: a warning card above the shelves, rendered only when the caller's queue is non-empty (hidden-unless-needed), linking to the page. - Resolve flow unchanged and already gated: MatchingHandler.LinkBook verifies device ownership and target-item visibility (403s) and creates the device file alias, so the next sync matches by path. SyncHandler.LinkBook and Handler.LinkBook are unrouted dead handlers — triage candidates, not part of the live flow.v1.16.1 |
||
|
John O'Keefe
|
9e831085ef |
fix(ui): derive Library Type dropdowns from library_types — audiobooks was missing
The audiobook library type shipped backend-complete (LibraryTypeAudiobooks constant, allowed extensions, MIME map, schema seed row, GET /api/libraries/types) but neither create-library form offered it: the admin Create Library modal and the setup wizard's add-library step both hardcoded their <option> lists to the original three types (ebooks/comics/manga), so an audiobook library could not be created from the UI. Both dropdowns now render from the same DB-backed library_types table the create API resolves types against — a future type appears without template edits (the exact drift the rendering-ownership rule warns about): - templates/utils.go: OrderedLibraryTypes pins the historical dropdown order (ebooks, comics, manga, audiobooks) and appends unknown future types by name; LibraryTypeDisplayName renders labels with a capitalize-the-name fallback for types added without a label - templates/admin_library.templ: new LibraryTypeOptions fragment used by the modal's select; AdminLibrary takes the types list - templates/setup.templ: Setup takes the types list, same fragment - internal/router/helpers.go: libraryTypeOptionsFor fetches via LibraryService.GetLibraryTypes, falling back to the built-in type names on DB failure so the forms stay usable (CreateLibrary still resolves the name against the DB, so the fallback is never authoritative) - internal/router/frontend.go, internal/router/setup.go: pass the list into the two pages Tests: - templates/utils_test.go: pure unit tests for ordering + labels - cmd/server/tests/admin_library_page_test.go: SSR regression — /admin/library must offer all four library types, audiobooks exactly once (options come from the DB loop, not a hardcode) |