Kobo emulation + KOReader calibre-compatible surface backfilled with
device-token auth; collections reorder + update_rule created.
Claude registry: 12 entries. kobo/koreader emulation endpoints all bind
arbitrary protocol POST bodies (markup/bookmark/gettests/sync-from-server,
koreader progress/bookmarks) — registered as AnyJSONObject, not bodyless as
first drafted. Initialization returns the library sync payload, not the init
response.
Contract correction in update_rule.md: the route is SetRuleEnabled ({enabled}
required -> {collection_id, is_smart}); Beam's draft invented rule-field
updates. Reordered response/status: {item_ids} -> {status: reordered}.
make test 9/9.
Beam backfilled 11 library docs (CRUD, folders, visibility, browse, stats,
media-items) and re-attached frontmatter to create_collection.md — which had
silently lost its pilot version in cc893f75: the negative-test
'git checkout --' restored the OLD file because the pilot was never staged,
and a frontmatter-less doc is invisible to the gate. Claude restored the full
corrected pilot content (priority higher-wins, time.String() note, 409).
Claude registry: 11 entries + one inverted catch — DELETE /:id/folders DOES
bind {folder_path} (handler reuses AddLibraryFolderRequest); registry first
draft said bodyless. GET /:id/visibility returns joined
GetUserVisibleLibrariesRow (type_name etc.), not bare Libraries.
Gate-driven merges fixed create/get library response shapes (sqlc row keys:
created_by_admin_id, library_type_id, updated_at; dropped invented type/
folders/is_visible). Fixed invented request fields (user_id in
set-visibility; library_type_id in update). make test 9/9.
Beam backfilled list/get/create/update/delete media-item, search, ratings
CRUD, progress GET/PUT. Beam correctly identified create/update/delete as
jwt+admin (admin group in the router — verified).
Claude registry: 12 entries. Notable pins:
- get_media_item: hand-built ~44-key map pinned verbatim from the handler
- create/update return the full database.MediaItems row (74 columns) —
examples merged up to the real shape via gate-driven merge script
- search returns []SearchMediaItemsRow (sqlc row — Real type pinned)
- rating scale is 1-10 int (odd = half-star); GET returns {rating: null} when
absent; update reuses the create handler (201)
- progress GET: 25 base keys + 4 CONDITIONAL position-authority keys
(css_selector/anchor_href/char_offset/context_text — added by the handler
for convertible formats with a valid CFI); PUT progress binds
sync.SaveProgressRequest and returns database.ReadingProgress
- Beam's examples were missing entirely on 4 files (list/search/progress
GET/PUT) — relabeled existing fences or inserted contract-shaped blocks
Gate-driven merge scripts (object + array variants) brought all examples to
the exact handler key sets. make test 9/9.
Beam backfilled auth core (login/register/refresh/logout/profile) and devices
core (list/update/delete/shelf CRUD/sidecar/register flow) with frontmatter,
provenance, and labeled examples. All claimed shapes verified against
handlers: AuthResponse {access_token, refresh_token omitempty, token_type,
expires_in, user}, update device {device_updated, device}, shelf wrappers,
DeviceAuthStatusResponse.
Claude registry verification: 15 entries pinned by type (LoginRequest,
RegisterRequest, AuthResponse, RefreshTokenRequest/Response, UserProfile,
DeviceListResponse, DeviceUpdateRequest, AddToShelfRequest, SidecarConfig,
DeviceRegistrationRequest/Response, DeviceAuthStatusRequest/Response).
Removed revoke_device.md — name-only duplicate of delete_device.md (same
DELETE /api/devices/:id route; was the last audit-orphan doc).
Gate catches this round (subtle Go serialization facts, now pinned in docs):
- uuid.UUID arrays never serialize as omitted: register/status response
ALWAYS carries device_id (all-zeros while pending) despite omitempty
- update device request binds device_name, not name
- register response includes device_id alongside registration_id
- list devices response is a {devices, total} wrapper
Beam added frontmatter + provenance + labeled examples to the 7 conflict and
6 queue docs, fixing real field drift on the way (resolve response keys,
bulk result shape, dismiss-all deleted count, queue stats counts).
Claude registry verification found TWO Beam catches of Claude's own draft:
- queue routes use :device_id / :item_id params (registry first draft said :id)
- resolve_conflict DOES take a body (ConflictResolutionRequest) — registry
first draft said nil
Both fixed; gate then surfaced one real Beam defect: example-request used a
nonexistent "resolution" field in resolve + bulk-resolve (real contract:
winner=<source|manual> with required manual_data for manual; strategy=
most_recent|highest_progress|manual). Fixed in the docs.
Pinned by type: ConflictListResponse, ConflictDetailResponse,
ConflictResolveResponse, BulkResolveRequest/Response, QueueStatsResponse,
ConflictResolutionRequest. dismiss-all deleted is an int count.
Beam drafted 15 frontmatter'd docs (saved-filters CRUD, dashboard
sections/preferences/restore, system config/settings, series list + books);
Claude verified shapes against handlers and pinned the registry:
- saved-filters: SavedFilterResponse pinned by type (RFC3339 timestamps here,
unlike most of the API); create/update share CreateSavedFilterRequest
(resource_type oneof media-items|collections|devices)
- dashboard: preferences GET pin {hidden_collections, collection_order,
items_per_section}; PUT returns full UserDashboardPreferences row; sections
wrapper {sections}
- system: GET+PUT /api/system/config are admin-defined key/value maps — new
AnyJSONObject sentinel (gate shape-checks without key comparison; request
side too). UpdateSystemConfiguration returns {status, message} — verified,
not guessed. GET /api/system/settings returns []SettingEntry.
- series: {series[{name, library_id, book_count, total_in_series,
cover_paths, last_entry_at}], total, limit, offset}; books by
?name=&library_id= with family-lens 403
Dedupe: deleted dashboard.md, system/config.md, system/settings.md — each
covered the same endpoints as Beam's per-endpoint files (content merged by
the gate-verified shapes; old prose recoverable from git).
Beam clarifications confirmed: scanner routes are WO5 (docs exist);
progress/books-match are WO5.
DeviceUsageResponse / PopularBooksResponse / services.JobResult pinned by
type; jobs create uses an inline anonymous request struct (RawMessage pin).
Two UNVERIFIED markers recorded (analytics date-range params, facet value
element shape).
New frontmatter'd docs:
- download (ServeFile — raw bytes, family-lens enforced; corrects the old
download doc's claim that no download route is registered: ServeFile IS
routed, DownloadBook is the dead code)
- set_audio_speed (speed range 0.25-4.0, NaN/Inf rejected)
- rescan (abridged example over the full 70+-column row; write-after-success
guarantee noted)
- purge-archived (destructive, admin), unarchive (204), merge (SHA-256
precondition, per-user collisions keep active copy)
- facets (two modes: facet+values, field autocomplete counts)
- filter_count (family lens)
Registry: RawMessage pins for map/abridged responses; Schema{} for non-JSON
body routes (download = raw bytes; unarchive = 204). Audit now:
missing docs down to ~55 (from 112), 0 orphans.
New frontmatter'd docs with real shapes (sqlc models pinned directly):
- bookmarks: get/create/update/delete (sync-aware rows: dedup_key,
last_modified_*, tombstones; delete is 204)
- review: upsert (empty string DELETES the review) / get (200 with
{review:null} when absent) / delete
- get single note + highlight (full sqlc row shapes)
- annotation history: restore (POST, kind via query or body) and purge
(DELETE, kind via required query; permanent)
Gate hardening forced by this batch:
- required-field walk now guards non-struct Request types (RawMessage pins)
- update_media_bookmark needed a full example block (no cross-reference
shortcuts — the gate checks every doc standalone)
New frontmatter'd docs with handler-compatible shapes:
- get_library_types.md (public; []LibraryTypes)
- list_libraries.md (admin; {data: ListLibrariesRow})
- get_library_folders.md (admin; []LibraryFolders)
- list/get stats/resolve/delete processing issues (admin; family-lens 403s
documented)
- scan_library.md (202 + 409 coalescing with active_job_id)
- get/update scan-settings (ScanSettingsResponse)
Gate improvements forced by this batch:
- Array responses: registered slice -> doc example must be a JSON array; key
set compared against the element type
- RawMessage pins may now be arrays
- update scan-settings uses a RawMessage pin: handler always sets Message but
the struct's omitempty would reject correct docs via zero-value marshal
Registry: sqlc models pinned directly (LibraryTypes, LibraryFolders,
ListLibrariesRow — json tags verified in the generated code).
Every API doc that declares frontmatter (method/path/auth/area) is now
checked by a unit test against three sources of truth:
1. Route registration: internal/router/*.go is parsed the way echo resolves
it (group prefixes concatenate; scanner.go's parameterized groups seeded).
Doc paths that no longer exist fail. (This is the check that would have
caught the /api/api/admin/hash-conflicts split.)
2. Auth: frontmatter auth label must match the middleware chain
(none|jwt|jwt+admin|device-token|optional-jwt).
3. Shapes: internal/docs/schema.go registry — example-request decoded with
unknown fields DISALLOWED (typo guard), required-by-validate fields must
appear (staleness guard), example-response top-level key set compared
both directions against the marshaled handler type (or a RawMessage pin
for hand-built map responses).
Five collections docs are the pilots with real handler shapes; notable
corrections they pin: update returns 201 (not 200), list excludes smart
collections unless include_auto=true, rule priority is 1-10 higher-wins
(docs claimed 1=highest), created_at is Go time.String() not RFC3339,
hide_finished/show_up_next toggles documented. CollectionResponse hoisted
out of GetCollections/GetBookCollections as CollectionSummary /
BookCollectionSummary so the registry can pin real types.
The gate fails on missing registry entries by design — that is the
maintenance contract. Known pre-existing (not this commit): vet copy-lock
warnings in media.go.
The admin group already sits under the /api prefix, so registering
"/api/admin/hash-conflicts" produced /api/api/admin/hash-conflicts while the
hash-conflicts page's htmx buttons posted to /api/admin/hash-conflicts/:id/
resolve — a 404 that htmx swallows silently (no swap, no error), so duplicate-
content resolution has been broken since the feature shipped (03cb4c78).
Verified live: sane path 401 anon -> 200 with admin token; doubled path 404.
Caller audit found no other consumers (Android app, koplugin, web reader,
tests, Bruno: no references). Docs already describe the sane path and are now
correct without edits.
Docs pages no longer embed the app Header (whose hx-post login form silently
degraded to a native GET, leaking credentials into the URL — this page never
loaded htmx). New docs-owned topbar: search, theme picker, Open App; anonymous
visitors get a plain /login link. Optional-auth middleware (hand-rolled —
echojwt's ErrorHandler return ends the chain, producing empty 200s) populates
the user for signed-in visitors.
Navigation rebuilt from the docs directory tree: 6 sections, subcategory
groups, server-rendered expansion and active states. Search results are a
templ partial (/docs/search/partial); code highlighting is server-side chroma
(previously dead hljs CSS); prose is theme-aware via typography DEFAULT vars
(prose-invert painted unreadable text under light themes). /docs renders the
index directly; standalone 404 without the tailwind CDN.
Tests: navigation routing/expansion (internal/docs), render pins asserting no
login form/password field/app sidebar ever return on docs pages.
lint-rendering: exempt *.test.ts fixtures (jsdom DOM building); docs.ts
baseline 5 -> 1.
- A progress sync conflict used to be flagged only inside the progress
card, far down a long page. An unresolved conflict now also renders
a full-width warning strip at the very top (under the app bar) that
opens the same resolution modal on click. Rendered from the same
ActiveConflict the progress section uses — absent when clean.
- Shortcuts dialog Mouse section gains an 'End time' row: click to
cycle total → book left → chapter left (§10's toggle was
undiscoverable).
- Undo: a genuine sleep fire now toasts with a Resume action (toast.ts
grows an optional action button; acting dismisses).
- Chain: 'End of chapter, then 15 min' — leaving the chapter starts a
15-min countdown instead of fading (SleepState.followUpMinutes).
- Clock: the popover gains a time input + Arm — armClockTime resolves
HH:MM to its next occurrence and reuses the minutes machinery.
- Dim: the ticker eases a pointer-events-none overlay toward 35% black
across a countdown's final 30s; stop/finish resets it.
(Fade-out and countdown labels already existed — §3 built them.)
audio_reader.templ's <head> was the only page head without the
favicon link — the tab icon vanished the moment you opened a
listener. Standard <link rel=icon> added, matching every sibling
page.
The Player settings popover gains a 'Default playback speed' row
beside autoplay — a compact rate select, always visible (the Speed ▾
footer hides when current == default, which made the setting
undiscoverable at 1x/1×); same setDefaultSpeed PUT. The shortcuts
dialog splits its pointer section into MOUSE (click/drag/shift/wheel)
and TOUCH (drag/pull-away) — the bar has two scrub systems and the
sections now say so.
Speed becomes a per-USER-per-BOOK preference that syncs across
devices, with a user-level default for books not yet played:
- new audio_playback_speed table (mirrors reading_speed's shape; a
preference, NOT a measurement and NOT a reading_progress column) +
users.default_audio_speed, schema in place
- chapters endpoint resolves per-book row → user default → 1.0 and
ships playback_speed + default_playback_speed (zero extra round
trips; default powers the dropdown marker)
- PUT /api/media-items/:id/audio-speed upserts (range-checked; the
client SPEEDS list stays the real gate; library visibility gates
writes)
- the user default rides the profile path like theme/timezone:
profile form gains a 'Default Playback Speed' select (server-side
SpeedOptions mirrors the web SPEEDS list), GET /profile carries it
- web: restorePosition applies the resolved server speed (localStorage
audio_speed demotes to a first-paint cache); setSpeed PUTs
fire-and-forget with a failure toast; the Speed dropdown gains a
'(default)' row marker and a 'Make N× my default' action with a
success toast
ALSO fixes a pre-existing profile-page bug surfaced by the §8 form
verification: the password section's current_password input was
required INSIDE the shared form, so HTML5 validation blocked every
Save Changes submission (dead since the shell migration). The
password flow keeps its own gates (disabled-button checkers + server
400).
Integration test TestAudioPlaybackSpeed pins the full contract (unset
→ 1x, upsert resolution, range rejection, default fallback for
row-less books, default updates re-satisfying unset books). Verified
live: 9 Playwright checks.
Alpine :style string bindings REPLACE the element's style attribute —
the rail's static background/border/z-index were wiped the moment the
binding evaluated, leaving bare text over the transports. Fold all
declarations into the single :style binding.
Clicking the end time cycles the three display modes, persisted per
device in localStorage (audio_time_display). Pure helpers (mode list,
normalize, cycle, endTimeLabel with 0-clamp and no-chapters fallback
to book remaining) + vitest; aria-label announces the current mode and
the next. Remaining values track the scrub cursor mid-drag. 140 unit
tests; 15 Playwright checks on KJV + Gardens.
Mouse: drag is free-flow 1:1 (rail only on Shift+drag, now with
window from/to edge labels); clicks land with pixel magnetism
(MAGNET_RADIUS_PX=10) — the old 5-second threshold was sub-pixel on
long books and never fired, which was the 'chapter scrub isn't
accurate' bug. Hover inside the radius locks the tooltip to the line
with a highlight. Shift+click jumps to the nearest line from anywhere.
Touch: drag is an incremental cursor (applyDrag) whose scale tightens
continuously as the finger pulls off the bar (dragScale, exponential
10^(-|dy|/80px)) — the rail is the feedback: window shrinks with the
zoom (touchRailSpan) and lifts toward the finger (railLiftPx).
Dialog retitled 'Shortcuts & interactions' with a Mouse & touch
section driven by POINTER_GUIDE (audio-keys.ts). Old seconds-threshold
snapToChapter deleted. 133 unit tests; 19 Playwright checks on the
KJV (1189ch) and Gardens (34ch) books.
The scrubber becomes a layered bar: the native <input type=range> stays
on top for a11y (focus, arrow keys, aria — its track goes transparent
via static page CSS) while the layers beneath render the state:
- Chapter ticks: density-decimated (denseTicks keeps only marks >= 4px
apart) so an 1189-chapter 86-hour book gets ~100 readable marks
instead of a solid mass; the current chapter's tick is tinted.
- Buffered span: the ACTIVE track's buffered-ahead end mapped into the
global bar (buffers are per-track and never span the whole bar).
bufferedAheadEnd reads the range containing the playhead — a huge
m4b's moov index at the file tail is its own buffered range and must
not read as 'the whole book is buffered'. Rides progress + the
events that demonstrably fire for Range-served files that go
network-idle while paused (timeupdate/canplay/loadedmetadata/playing).
- Wheel over the bar: ±5s nudge, Shift = ±60s.
- Hover tooltip: time + chapter title under the pointer.
- Precision rail: while dragging, a ±5-minute window floats below the
bar (fixed anchor; release and re-grab to travel far) and the drag
maps at minutes-per-pixel instead of hours-per-pixel — the whole
point for 80h+ books. Click snaps to a chapter line within ~5s
(snapToChapter).
Pointer handling is ours end-to-end: pointerdown maps hours-per-pixel
+ snaps, captured moves remap through the rail, release seeks. The
native drag default is prevented so the two mappings never fight;
keyboard arrows keep their native step and seek via change.
Time labels get reserved width — the bar no longer resizes when the
elapsed label grows from 0:00 to 43:07 mid-drag (caught by the rail
drag check measuring a 25% mapping drift).
Pure math in audio-scrubber.ts: pointer mapping, chapter lookup, snap,
tick density, per-track buffer→global span, rail window/mapping/ticks.
Unit: 16 new tests including both book shapes (86h/1189ch and
26h/34ch). Live: 23 Playwright checks on KJV (folder) and Gardens of
the Moon (single-file m4b); zip books share the folder manifest shape
(no zip fixture exists in the dev library). Probe user deleted.
Header gear (beside the keyboard icon) opens a 'Player settings'
popover — one option row today, structured for more: 'Autoplay on
open', a role=switch toggle persisted to localStorage audio_autoplay
('1'/'0', default off) via the normalizeAutoplay one-source helper.
maybeAutoplay() runs strictly AFTER restorePosition resolves: the
?loc=audio:<seconds> deep link keeps winning on position; the setting
only decides whether playback starts. A browser autoplay-policy
rejection (or any play() failure) is swallowed silently — the user
just taps play; the unhandled-rejection trap in init would otherwise
shout 'REJ:' into the title. settingsOpen joins closeTopSurface, so
Esc closes the panel with the other surfaces.
Unit: normalizeAutoplay tests. Live: 13 Playwright checks — default
off stays paused, toggle + persistence, reload autoplays with no
gesture, deep link lands at 600s global while autoplay starts, off
stays paused, and a stubbed NotAllowedError proves the silent
swallow; probe user created and deleted.
Off / 5 / 10 / 15 / 30 / 45 / 60 minutes / end-of-chapter. The moon
pill sits in the settings strip between Bookmark and Volume and shows
the live mm:ss countdown while armed (or 'End of chapter'); the
dropdown's armed row highlights like Skip/Speed.
audio-sleep.ts is the pure, DOM-free state machine (arm/due/fired/
fade-ramp/label + the SLEEP_CHOICES vocabulary the templ dropdown
iterates). The component owns the timers: a 1s countdown ticker in
minutes mode, and a ~5s element-volume fade on expiry that ends in
audio.pause() — the SAME pause listener a manual pause fires, so the
progress push and media-session state land. The user's configured
volume is restored right after the pause, and the fade operates on
the element's volume the user controls, not a parallel path.
End-of-chapter fires when the armed chapter is left — natural
crossing (tickChapter) or a manual jump (jumpToChapter) alike. Manual
play during a fade aborts the expiry (you're awake, you asked);
stopSleep restores any partial fade. Per-session ONLY: nothing
touches localStorage — the reload check proves it starts Off.
sleepOpen joins closeTopSurface's dropdown branch, so Esc closes it
with the other settings-strip surfaces. Muted (or zero-volume) expiry
skips the pantomime and pauses immediately.
Unit: audio-sleep.test.ts (deadline mechanics, chapter-fired
semantics both directions, fade ramp clamps, label formatting,
SLEEP_CHOICES integrity). Live: 24 Playwright checks on the KJV book
— dropdown, countdown ticking, armed-row highlight, end-of-chapter
fire + reset, internal 3s deadline fade→pause→restore, play-abort,
reload-starts-Off; probe user created and deleted.
Space/Enter (body-focus only), H/L + arrows skip, J/K + PgDn/PgUp
chapter steps, B bookmark capture (note box auto-focuses, Enter saves
even when blank, Shift+Enter inserts a newline, Esc cancels),
Alt+arrows / Alt+H/L speed steps, up/down volume with a brief OSD,
M mute, [ ] skip-duration cycle, ? shortcut dialog, Esc closes the
top-most surface (help → capture → note edit → drawers → dropdowns).
audio-keys.ts is the ONE declarative table: resolveShortcut() drives
the document keydown dispatcher and visibleKeymap() renders the
templ dialog (x-for over the exposed keymap — templ cannot import
TS). Hardware media keys ride the existing MediaSession handlers;
the Media* keydown codes resolve through the same table to the SAME
methods, so both layers stay one behavior.
Guards mirror the ebook reader: typing in any
INPUT/TEXTAREA/contentEditable never triggers shortcuts (bookmark
notes included), a focused button keeps Space/Enter for itself, and
the focused scrubber keeps its arrows (its own @change seeks — a
native arrow nudge moves 0.08s, not a 10s skip).
SKIP_SECONDS consolidates to one const beside SPEEDS (the templ skip
dropdown now iterates skipChoices; [ ] cycles via cycleSkip()) — the
same one-source pattern as the §5 speed-list consolidation.
Unit: audio-keys.test.ts (resolution, Alt disambiguation, media
keys, keymap integrity and conflict-freedom, dialog visibility) and
cycleSkip tests. Live: 39 Playwright checks on the KJV folder book
covering every binding, the guards, dialog and OSD; probe user
created and deleted.
The rate list was duplicated — hardcoded in the templ x-for AND
implicitly in audio.ts — and phase 2's keyboard stepping would have
needed a third copy. SPEEDS is now the single exported const (the
agreed nine values, 0.5 through 3); the component exposes it as
'speeds' for the dropdown's x-for (templ cannot import TS), and
stepSpeed() walks the same array for the future audio-keys consumer.
normalizeSpeed() maps persisted values onto the list: garbage or
out-of-list localStorage entries fall back to 1x, and init() writes
the normalized value back so storage heals instead of staying
poisoned.
Pure logic unit-tested in audio.test.ts; live-verified with Playwright
on the KJV folder book: nine options render, pick/reload/persist and
the stale-value fallback all behave, button label and
audio.playbackRate agree in every case.
restorePosition() swaps the <audio> element's src (manifest track for
multi-file books, whole file for single-file, plain URL on offline
start), and loading new media resets playbackRate to the element
default. init() had applied the persisted speed before any src
existed, so after the swap every load actually played at 1x while the
Speed pill still showed the stored value — pick 1.5x, reload, and
audio was back at 1x.
The seekTo() src-swap path already re-applied the rate; the initial
load paths now do the same (manifest branch + offline-catch branch).
Surfaced by the phase-1 speed-list browser check: button label and
audio.playbackRate disagreed after reload.
Codify the standing rule at the top of the Tests section: the dev DB is
the test environment, and the invariant every test must uphold is the
testuser experience — its settings, scans, and reading data must be
exactly as the suite found them. The helpers exist to enforce this;
when a test needs protection the helpers don't provide, expand the
helpers rather than hand-rolling.
New known-gotchas from the 2026-10-21 incident cluster:
7. scanner re-extraction is write-after-success — audio_metadata is
guarded on both writers; new scan-owned column writes need the same
guard.
8. templ double-escapes style attribute values (sanitizer + EscapeString)
— emit quote-free CSS via unquoted url() tokens through SafeCSS.
9. vp test runs the vite config — build-hook plugins fire in the test
context too; outDir-mutating plugins need an "am I a build?" guard.
The testuser-experience rule: the suite must leave shared settings
exactly as it found them. setSystemSettingRestored snapshots the prior
system_settings row, writes through the sanctioned API path (the only
one that reloads the settings registry — raw SQL leaves the cache
stale), and registers a cleanup restoring what WAS there, never a
hardcoded value.
setExtractorVersion now rides it, so the ev-pin tests no longer
restore scan_extractor_version to a hardcoded 1: a deliberate dev bump
of the extractor version survives a suite run instead of being
silently cancelled (the live dev server obeys that setting on its next
poll and would stop re-extracting parser fixes). The now-redundant
explicit t.Cleanup restores are removed from
TestExtractorVersionReextract and pinExtractorVersionOne.
Three tests in scanner_audio_guard_test.go pin the routes the wipes
traveled; each drives a real scan/rescan/edit over ffmpeg-rendered
fixtures and asserts the payload survives with duration and manifest
intact:
- TestRescanMediaItemPreservesAudioMetadata: folder book through the
focused payload writer.
- TestRescanMediaItemPreservesAudioMetadataSingleFileM4B: the incident
class verbatim — single-file m4b (no layout key) routes through the
generic updater; fixture rendered with ffmpeg AAC/ipod
(writeTestTrackM4B).
- TestMetadataEditorSavePreservesAudioMetadata: the metadata editor's
full-row save through PUT /api/media-items/:id.
Verified to bite: with the handler fix stashed, the editor test fails
with "metadata edit must never NULL audio_metadata"; with it, green.
PUT /api/media-items/:id (the metadata editor) writes the FULL row via
UpdateMediaItem, whose query sets audio_metadata unconditionally — and
audio_metadata has no editor field, so the params literal left it nil
and every save NULLed the payload. A plain metadata edit (genre tweak,
title rename) dropped any audiobook back to "1 second long" with no
track manifest: the same incident class as the scanner wipe, sitting
on the most-used admin path in the app.
Carry existing.AudioMetadata through the write. Write-after-success:
only the audio chain (probe-backed extraction /
UpdateMediaItemAudioBookPayload) ever replaces that column.
The 2026-10-21 wipe: a re-extraction pass NULLed audio_metadata on the
m4b audiobooks — every affected book dropped its chapters, track
manifest and duration and displayed as 1 second long (the reader also
lost its manifest, since single-file books synthesize it from the
payload).
Guards on both writers of that column:
- updateMediaItemWithMetadata (generic full-row pass): when the new
extraction carries no audio payload, keep existing.AudioMetadata.
Single-file m4b books route here, and a failed/absent extraction
leaves AudioMeta nil.
- updateAudioBookItem (focused payload write): an empty payload (no
tracks, no chapters, no duration) over a populated row is a failed
extraction — refuse the write with a loud warning and let the next
scan retry.
audioPayloadIsEmpty() pins the empty-payload predicate (unit-tested):
the column is only ever replaced by fresh data from a real audio
extraction, never by absence of one.
manifest.json has referenced /static/icons/icon-{192,512}.png since
April, but the directory was never committed — ~100 404s per browsing
session and a broken PWA install. Rendered both sizes from the
existing favicon book glyph on the Tokyo Night palette
(#1a1b26 background, #7aa2f7 stroke).
audio_reader.templ was the only template emitting <!DOCTYPE html/>
(trailing slash is a parse error), which put the audio player page in
quirks mode — document.compatMode was BackCompat while every other
page is CSS1Compat. Quirks layout rules are a standing hazard for the
player's h-screen/flex page even where it happens to render. Drop the
slash.
templ's codegen wraps style attribute expressions in a sanitizer AND
templ.EscapeString, so any quote character — even inside templ.SafeCSS
— reached the DOM as literal ' text. The hero backdrop's url('...')
never parsed: no blurred cover on any book detail page, and Firefox
resolved the mangled token as a relative URL, fetching /media/& (500)
once per page load.
Fix: emit quote-free CSS — an unquoted url(...) token built through a
new cssURL() helper that percent-encodes the characters that would
break it (space, quotes, parens, backslash). Stored cover paths are
already percent-encoded URL paths, so the escaping is defensive only.
Clicking the bottom progress slider left it focused, and the reader's
keydown guard treats INPUT focus as typing — so arrow keys stepped the
range input instead of turning pages. With step="any" each press moved
~1% of the book and key repeat crossed the whole book in seconds; the
debounced progress saves then persisted every wild landing (positions
scattered from page 5 to 285 to 100%, "progress no longer saves
correctly after page turns").
Two layers:
- reader.templ: all three progress sliders blur on pointerup, so a
mouse scrub no longer leaves focus behind.
- reader.ts handleKeydown: ArrowLeft/ArrowRight on a focused progress
slider preventDefault, blur, and fall through to the page-turn
handlers; every other key still belongs to the control, so a
keyboard user who Tabbed to the slider keeps arrows for it.
The cleanStaleChunks plugin lives in the shared vite config, so its
closeBundle hook also fired under `vp test` — where generateBundle never
runs, produced stayed empty, and the sweep deleted every hashed chunk
in web/static (28 files → 3). The entries still referenced their old
imports, so every page 404'd and Alpine never started: the whole reader
and media system appeared dead after merely running the test suite.
Guard: skip the sweep when produced.size === 0 — not a build run, no
build output to clean. Verified: build → 28 files, test → 28 files;
back-to-back builds stay byte-identical.
Test infrastructure uses the vite-plus stack: vp test (bundled Vitest 5),
jsdom environment configured in vite.config.ts, colocated web/src/*.test.ts
files that tsc type-checks, vp fmt formats, and the bundler never imports.
- markdown.test.ts pins the notes XSS defenses: parser-level html escaping,
DOMPurify FORBID_TAGS, scheme checks, footnote rendering, html->markdown
round-trips incl. the headingless-table rule.
- filter_builder.test.ts pins the deep-link wire contract (ruleToFlatParam),
smart-serialization rules (isFlatable), and pruneIncomplete semantics.
- password_validation.test.ts covers the rules fetch + fallback and the
full submit-gating matrix through the real DOM flow (debounce included).
- api.test.ts pins response/error contracts incl. the HTTP-status and
Unknown-error fallbacks and the 404-as-null progress contract.
- storage.test.ts covers token/theme/library round-trips, the __all__
sentinel, and the SSR cookie.
- toast.test.ts pins config completeness for every ToastType (the missing
'warning' entry regression) and render behavior.
Found + fixed while writing these: the no-useless-escape fix in
d7d14957 had de-escaped the ']' in hasSpecialChar's character class,
closing it early — special-character validation silently stopped
matching most specials. The new password suite caught it immediately.
jsdom chosen over happy-dom after empirical comparison: DOMPurify's
sanitizer output is mangled under happy-dom (forbidden tags survive),
which would have made the security suite test nothing.
Note: AGENTS.md gains a 'Web (TypeScript) tests' section documenting the
workflow and the happy-dom gotcha.
Matches the error/success/info entries visually. bg-yellow-500/90 is
generated by the Tailwind scanner from this file itself (content globs
include web/src/**/*.ts), same as the sibling bg-*-500/90 classes — no
safelist entry needed.
tsc --noEmit and oxlint web/src are now both clean; vp check passes.
- htmx: one canonical Htmx interface in shims.d.ts (ajax/process/trigger
per the htmx 2.x API). alpine.ts had a second, conflicting Window.htmx
declaration (TS2717) — removed; bookPicker/collections lose their
(window as any) escapes now that Window.htmx is properly typed.
- tsconfig lib ES2020 -> ES2021 + DOM.Iterable: replaceAll exists
(escapeHtml) and for..of over querySelectorAll results is legal.
Type-checking only — vite emits at esnext regardless.
- custom-section-builder: selectedBooks holds identity fields only;
typed as Pick<BookInfo, ...> instead of forcing partial API shapes.
The API-critical BookInfo contract is untouched.
- toast: real bug found by the type system — setup.ts passes 'warning'
but getToastConfig had no entry, so the warning toast threw TypeError
at runtime instead of displaying. Added the type + config. Uses
bg-yellow-500 (verified present in scanned templates => generated by
Tailwind); the one intentional behavior change in this commit.
- book-detail: instanceof narrowing replaces an as-cast; unused
AnnotationSource import removed.
- password_validation: explicit HTMLElement | null for the checklist
list (querySelector('ul') infers HTMLUListElement, conflicting with
the container-as-list branch).
- stats: tooltip parsed value typed via annotation (bar=scalar,
line={x,y}); dead getActiveRange() removed (was already tree-shaken
out of every bundle).
- library-switcher: onSwitch honestly typed void | Promise<void>
(series.ts passes a sync navigator).
- 20 unused catch bindings converted to optional catch binding.
Verification: fresh build is byte-reproducible; every chunk is
identical to the pre-fix build except main.js (instanceof narrowing,
regex fix, stats refactor), reader.js (destructure), and the shared
storage chunk (renamed — carries toast.ts). reader/main diffs audited
hunk-by-hunk against the intended edits.
The extractor/force-rescan tests assert manifest stamps against the
default version (1), but the setting lives in the SHARED dev DB — any
prior run (or a manual admin-UI bump) can leave it at 2, failing every
stamp assertion through no fault of the code. Each ev-sensitive test
now pins the setting to 1 at start and restores 1 on cleanup; proven
against a deliberately bumped setting (tests pass, row restored).
Force rescan contract change (owner-directed): the editorial-keep skip
is retired. Force rescan now re-derives EVERY field on EVERY book from
all sources — multi-file books included (the folder/zip flows ran the
full chain on create and on manifest-flagged edits, but skipped the
sidecar apply under force). metadata_overrides is the only protection:
ApplyMetadataOverrides re-applies manual values inside the writer, so
manual edits survive; everything else re-derives.
- A force pass also STAMPS each re-derived book's manifest with the
current parser version (success-only — same truthfulness rule as
Phase 10) and closes its open processing issues (Phase 11 self-heal).
Force rescan and the ev-bump trickle now converge: either leaves the
library fully current; the deploy order is bump-first, then Rescan
All (documented in the plan and contract_pins).
- Single-file force rescan already ran the full chain — unchanged,
now pinned.
NEW filter field manually_edited ("Manually edited", boolean):
cardinality(mi.metadata_overrides) > 0 / = 0. Composed-engine (qf)
only — the builder's Book facts group gains it with is true/false;
no value menu, no suggestion surface; family lens inherited.
Tests: multifile force applies sidecar metadata + stamps + override
protection; single-file pin; the filter both directions scoped to the
test library; adjacent suites (Audiobook, Calibre, ExtractorVersion,
CrossLibraryMove, ProcessingIssues, FilterBuilder, SchemaIdempotency)
green. The extractor mechanism itself is untouched — the owner bumps
scan_extractor_version manually via the admin UI (plan Phase 13 §3).
processing_issues only recorded cross-library move rejections; files
that failed to import vanished into an error counter and stdout. The
ledger now records every failure class and heals itself:
Schema (in place, pre-release boundary):
- media_item_id NULLABLE + file_path TEXT (deliberately unbounded —
an over-long path is itself a failure mode; a VARCHAR(500) ledger
column died on the exact 22001 it existed to report — caught by the
new test). Row-less failures key on (library, file_path); item-backed
on (item, type). Widened dedup = COALESCE-expression unique index;
the old table UNIQUE stays (NULLs pairwise-distinct for row-less).
- Conflict-target upsert: repeat failures UPDATE the same row and
REOPEN resolved ones — never duplicates.
Wirings:
- CreateMediaItem failure (single-file + multifile createAudioBookItem)
→ row-less import_failed (error).
- Extraction hard-fail with fallback import → item-backed
metadata_extraction_failed (warning).
- Change-path re-extraction failure (the ev/sidecar guard) → item-backed
metadata_extraction_failed; LogProcessingIssue keeps its signature, a
row-less helper joins it.
Auto-resolution:
- Successful import → ResolveOpenProcessingIssuesForPath (library,
file_path).
- Successful (re)extraction → ResolveOpenProcessingIssuesForItem
(single-file change path, multifile editorial apply).
- Unchanged-skip scans do NOT resolve: extraction never re-ran, so the
degraded metadata is still degraded — the page heals exactly when the
book does.
Family lens on ALL issues surfaces (list, stats, resolve, delete, SSR
page): explicit out-of-set library id → 403, never silent. This
tightens one pre-lens behavior — a nonexistent library id returned
200-empty, now 403 (pinned, changelog entry in docs/contract_pins.md).
List is LEFT-JOINed so row-less rows appear (media_item_id "",
title falls back to the path).
Tests: full lifecycle state machine through real scans (fail → appears,
repeat → no dups, success-at-path → resolved, LEFT-JOIN list shape);
family lens 403s with a fresh non-shared admin; all 7 pre-existing
issue suites green (two expectations widened to the lens, changelogged);
adjacent suites (CrossLibraryMove, Audiobook, Calibre,
ExtractorVersion, SchemaIdempotency) green; unit suites green.
setupTestServer built the settings registry (and wired it into
systemSettingsHandler) but never called authHandler.SetSettings, unlike
main.go:112. The auth handler therefore ran with a nil registry and
defaultTimezone() took its nil-fallback branch, returning "UTC" for every
Register/AdminCreateUser in tests regardless of the system default_timezone
setting — so the 740213f0 inheritance pin in TestAdminCreateUser could never
pass and failed every full-suite run.
Also replace the TestAdminCreateUser "stable-ish" uniquifier (MaxConns is
pinned to 1 by the harness, so it produced the same user every run and
depended on the next run's cleanup sweep) with a real uuid suffix.
Parser fixes (7e4c97f6, genre model v2) never reached rows ingested
under an older parser: the sidecar never changed, so every fingerprint
said 'unchanged'. The manifest JSONB now carries the reserved 'ev'
stamp — the parser version that produced the item's row.
Semantics (all pinned by tests):
- ABSENT ev decodes as 1 ('v1-or-unknown'); stamping is truthful-only:
create paths + post-extraction stores stamp the setting's current
value; deploy-day backfill and the extract-failure guard stay UNSET
(a row never claims a version its data can't prove — a mid-backfill
bump self-heals next poll); repoint reconciliation preserves the
stored ev (stat-only refresh re-extracted nothing).
- NEW setting scan_extractor_version (seeded + registry + getter,
int default 1, category scanner): a manifest whose ev != the setting
re-extracts through the SAME change path (full extraction →
updateMediaItemWithMetadata → fresh stamped manifest). Sidecar-less
books get an entries-less manifest as their ev home (create: stamped;
first-sighting: unset).
- Catch-up bounded: evReextractBudgetPerScan=100, atomic (watcher pass
can overlap a poll scan), reset per ScanFolders — amortized +
resumable across polls; no force rescan, no flood.
- Multifile books included: the ev gate rides multifileSidecarChanged's
shared ladder; ev-driven manifestChanged drives the editorial apply
+ stamped re-store.
Settings API: the unified GET/PUT /api/system/settings now lists the
key (registry-driven, auto-surfaces in the admin UI). Test harness:
the registry is now Loaded + wired into the SystemSettingsHandler
(main.go parity) — settings writes through the API were 503s before.
Verified: unit (ev normalization, budget CAS, GetExtractorVersion
cache paths, registry clamping); integration state machine —
create-stamp ev=1 → equality skips (corruption probe persists) →
wiped scan-state backfills UNSET without re-extract → bump re-extracts
ONCE (sidecar value restored, ev=2) → stable (second corruption
persists); multifile twin green; adjacent suites (Audiobook, Calibre,
CrossLibraryMove, SchemaIdempotency) green; live app restart re-scans
1198 files with 0 changed (deploy-day rule), setting listed at
/api/system/settings.
John O'Keefe