A bare string-prefix test (strings.CutPrefix / strings.HasPrefix) let a
registered folder whose path was a string prefix of a sibling claim the
sibling's files: "/app/audiobooks" matched "/app/audiobookssd/Brian
Jacques/...", so getRelativePath sliced the remnant "sd/" into every
stored relative path for that library (prod scan log: "[AUDIO] Created
folder book 'Taggerung' (41 tracks) at sd/Brian Jacques/Redwall/Book 14").
Scanner lookups built their keys with the same function
(getMediaItemByFilePath, scanStateLookup), so writes and lookups were
consistently poisoned and no scan ever corrected them; everything
outside the scanner resolves paths by joining back to the roots
(ResolveMediaPath), so cover/track URLs 404ed for files sitting one
directory away.
The same unchecked HasPrefix pattern also drove five folder-membership
sites, so a sibling-prefix pair misattributed more than paths:
owning-library resolution (processMediaFile, maybeProcessAudioBook),
per-library-type extension allowlisting (isScannableFile), and
folder-structure metadata extraction on both create and update paths —
a file could be scanned under the wrong library's rules entirely.
folderContains(folder, path) is now the single boundary-aware
primitive: path must equal the folder or continue with a
filepath.Separator (after Clean). All seven sites go through it, so no
folder name — however similar to another — can interfere with any
other's files again, in any order of registration; trailing slashes on
registered folders are normalized. Registration order still decides for
genuinely overlapping (nested) registered roots: pre-existing behavior,
deliberately unchanged.
Regression tests (media_scanner_path_test.go, pure unit — no DB) pin the
prod trap (audiobooks/audiobookssd), order-independence, trailing
slashes, mid-word siblings, and the helper itself.
Repair of already-poisoned rows is deliberately separate and safe only
under this fix: a rescan repoints them via the SHA-256 move path
(MoveMediaItemFilePath) once lookups stop producing poisoned keys.
The "top search bar doesn't work" report had TWO stacked causes, found
by reproducing with real keypress-by-keypress typing instead of
Playwright's fill() (which fires a single input event and masked both):
1. htmx.min.js was included PER PAGE. The header's live search is
htmx-driven on every page, but five pages rendering @Header —
series, admin, admin hash-conflicts, admin processing-issues,
browse-detail — never loaded it: the input's hx-* attributes were
inert (htmx-internal-data absent), so typing produced no request at
all. The include now lives in the header partial itself (classic
script, before the module bundle); the sixteen per-page copies on
@Header pages are removed (double-processing guard). Non-header
pages (login/register/setup/reader/index) keep their own.
2. htmx's delay: modifier on a PROGRAMMATICALLY dispatched custom event
(htmx.trigger) loses the request entirely under rapid successive
triggers — every re-fire canceled the pending timer and the last
never issued, so fast typing (the normal kind) never opened the
dropdown while one-shot fill() worked. The debounce moved into the
Alpine component (client-local state, where it belongs); htmx now
executes the bare `search` event immediately.
Also made bundle staleness structurally impossible: each bundle's
script stamp is a content hash computed at startup (templates/
bundles.go), so a rebuild that changes bundle bytes changes its cache
key with no manual ?v=N bump — the manual stamp was how the first
dropdown fix stayed invisible for a deploy. The reader bundle (never
stamped at all) is covered too. HTML pages additionally send
Cache-Control: no-cache (they are dynamic), so a stale page can no
longer reference a stale stamp; /api, /static and /uploads keep their
native caching, pinned by a middleware test.
Two follow-ups from live testing:
- "recent" is BOTH a base toolbar key and a full registry key (where it
means desc). ResolveSeriesSort's full-key passthrough ran first, so
sort=recent&dir=asc silently resolved back to desc — the direction
chevron flipped but Date Added could never sort ascending. "recent"
now composes with the direction before the passthrough, unknown bases
fall to the default entirely (a hostile dir can't ride an unknown
sort), and the composition table is unit-pinned plus an HTTP-level
row in the series filter test.
- The header search "still broken" report was a stale-cache artifact:
the fixed code shipped but main.js kept its ?v=7 stamp, so browsers
with a cached copy never refetched (fresh-profile e2e passed while
the user's browser didn't — heuristically cached module). Stamp bumped
to ?v=8 in header + docs; the e2e suite re-verified against the
rebuilt server.
Per the owner's testing bar: a Playwright spec registers an admin,
seeds one fixture per write-capable type (EPUB, PDF, CBZ, MP3, FLAC,
M4A, M4B — generated at runtime, no binary media in git), drives the
REAL metadata-editor flow (Edit → change title → in-app dialog →
Save & write) for each, and then reads every artifact back
INDEPENDENTLY of the app — EPUB/CBZ/sidecar via python zip+XML parse,
audio via ffprobe — asserting the written title landed in the FILE and
the old one is gone.
Writing the e2e exposed a real bug: atomicWrite left server-written
sidecars at 0600 root (os.CreateTemp's default), so host-side tools
sharing the volume — Calibre reading metadata.opf — were locked out.
The temp file is now chmod'd 0644 before the rename, matching the
scanner's cover-file convention; the e2e's host-side reads prove it.
Three defects from real-world use:
- The m4b rewrite assumed every stream survives -map 0 -c copy; files
carrying text-ish tracks (the movenc 'text' fourcc) fail the ipod
muxer with "Tag text incompatible with output codec id" — exactly the
error a write attempt hit in the field. The rewrite now attempts full
fidelity first, then retries mapping audio + attached pictures only
(-dn -sn): chapters live in the container header (-map_chapters 0)
so they survive; every attempt still ffprobe-validates before the
atomic rename and the original is never touched on total failure.
- The browser confirm() is replaced by an in-app dialog
(WriteToFileConfirmDialog, Alpine + templ — no native dialogs).
- "Write to file" now SAVES the pending editor edits first (persisted
through the same PUT the Save button uses), then writes the file
from those DB values, and closes the editor + refreshes the page
exactly like Save — previously the endpoint wrote the LAST-SAVED DB
values, so edits sitting in the modal never reached the file, and
the page didn't visibly update.
Test coverage per the owner's bar: EVERY single-file container (MP3,
FLAC, M4A, M4B) gets its own runtime-generated fixture, gets written,
and the FILE's metadata is then verified independently — ffprobe reads
the rewritten container and every mapped key is asserted per format,
including the subtitle round-trip split (MP3/FLAC yes, M4A/B no) — and
the EPUB/CBZ/sidecar assertions now PARSE the written XML (dc titles/
creators, ComicInfo fields, calibre meta) instead of substring
matching. A smart-genre unit test also pins the mixed-blob contract: a
recognized shelf wins the genre column even when demoted format words
ride along in the same blob.
Toolbar feedback round: the series search box now matches whatever the
main search matches for a group — series NAME, any book's AUTHOR, and
any book's TITLE inside the series (WHERE EXISTS per group-row, so
counts stay true); the separate author input is gone from the toolbar
(the author= param stays live for the JSON API and deep links). The
sort control becomes a base-key select (Date added/Name/Author/Books/
Progress/Last read) plus a chevron direction toggle that flips and
resubmits; sort/dir compose through ResolveSeriesSort (full registry
keys keep working for back-compat, unknown base keys fall to recent).
The search/apply button moved next to the search box as requested, and
default URLs stay clean (sort/dir only emitted once the user chooses).
Phases 5b and 5c of the metadata writer.
5b — single-file audio (MP3/FLAC/M4A/M4B): the container's tags are
rewritten with ffmpeg -c copy (no re-encode; -map 0 keeps cover art,
-map_chapters 0 keeps chapters). The key set is exactly what the
scanner's applyAudioTagFallbacks reads back — title+album, subtitle,
artist/album_artist, composer (narrators, Audible's convention),
grouping + iTunes series-part, date, genre — so a write + rescan
round-trips. Output is ffprobe-VALIDATED (duration within 1s/0.5% of
the source) before the atomic rename; a lossy container rewrite never
replaces a good book. Fields the format can't carry per our own read
chain are deliberately not written, and fields_written reports only
what really round-trips (subtitle rides MP3/FLAC — this image's mp4
muxer drops subtitle keys, discovered empirically and pinned by test).
5c — CBZ: ComicInfo.xml injected (or replaced) with the same atomic zip
rewrite as the EPUB path, extended with an append-if-missing mode
since a comic may not carry one yet. Field map mirrors the scanner's
comic block: Title/Series/Number/Count/Volume, Writer (authors),
Summary, Publisher, Year/Month/Day, LanguageISO, Web, Genre+Tags;
fractional series numbers are omitted rather than truncated to a lie.
CBR/CB7 stay on the sidecar path (RAR/7z can't be rewritten in place).
Integration tests generate media at test runtime with ffmpeg (m4b) and
in-memory zips (cbz) — no binary fixtures — pinning the tag round-trip
through ffprobe, post-write rescan titles, the ComicInfo injection and
the validation-gate behavior.
Phase 5a of the metadata-writer feature: the book editor gains a
"Write to file" action (admin) that persists the item's CURRENT
database metadata into the artifact on disk, so the library stays
portable to Calibre and other apps — the thing Save alone can't do.
Dispatch (services/metadata_writer.go): EPUB/KEPUB gets its embedded
OPF spliced in place (fresh dc:* block generated inside the ORIGINAL
metadata open tag; manifest/spine/guide untouched; every other zip
entry copied byte-identical in original order so mimetype stays first
and STORED; original dc:identifiers preserved — the package's
unique-identifier must keep resolving). Everything else takes a
Calibre-style sidecar: metadata.opf when the book is ALONE in its
directory (Calibre's own convention — round-trips through Calibre
too), <basename>.opf when siblings share the folder, so one book's
sidecar can never poison dir-mates' scans (the scanner now reads both,
basename first).
Safety contract: temp file + atomic rename — the original is never
truncated on failure; the generated OPF is unit-tested to ROUND-TRIP
through the scanner's own parser (write side and read side can never
disagree); metadata_overrides are deliberately kept (file and DB agree
after the write; overrides still guard against stale sidecars). The
endpoint re-extracts inline (RescanMediaItem) and reports rescan
failure separately — a failed rescan doesn't undo the write. SHA-256
changes are inherent and documented for users (Kobo/KOReader re-key).
UI: the metadata editor modal carries the file path as a data
attribute; the button confirms before mutating, labels the target
(embedded vs sidecar) and toasts the outcome. Integration tests pin
the 403, missing-file error, EPUB embed + post-write rescan title, the
lone-book metadata.opf case, and the shared-dir <basename>.opf case
with no dir-scoped litter.
Search results now show where you are in a book: a completed check at
the 0.999 threshold (same cut as the filter status arms), a partial
progress bar along the cover while mid-read, nothing when you never
opened it. Rendered in BookCard (any grid whose adapter loads viewer
progress) and in the header-search dropdown rows.
The progress rides a LEFT JOIN reading_progress on the SAME user_id
param SearchMediaItemsUnified already binds for library visibility —
one requester param, no new wire inputs, row-neutral join (the table
is UNIQUE per media_item+user). SearchMediaItemsUnifiedRow and BookInfo
gain the keys additively (viewer_percentage null when unknown,
library_name empty on feeds that don't load it); contract_pins.md
carries the changelog entry and the pin tests widen in lockstep.
The header dropdown itself migrates off the innerHTML string renderer
in search.ts (a rendering-ownership violation: a templ twin existed via
the HX-Request branch). The input now hx-gets layout=dropdown and swaps
the new templ SearchResults partial — titles/authors highlighted
server-side (escape-first, safe by construction), rows with the same
progress overlays — while a slim Alpine component owns only
client-local state (min-length gating, open/close, keyboard selection,
htmx lifecycle). web/src/rendering-allowlist.txt drops the search.ts
baseline to zero accordingly.
Integration test pins the per-user join (fresh user sees 1.0/0.42/null
on the fixture; the shared admin sees null everywhere — read-only) and
the dropdown partial's overlays/count/row count.
The /series page rendered its grid in one fixed order (newest first)
with no way to find a series beyond eyeballing cards. The page now has
a toolbar — series-name search, author filter, viewer-progress filter
and a 12-key sort select — wired as a plain GET form (the page's
existing model: the URL is the single source of truth, and the library
switcher now carries q/author/status/sort along on switch).
Extensibility is deliberate (user request): sorts live in a registry
(services.ResolveSeriesSort) mirrored by a CASE cascade in
GetDistinctSeries and a <select> in series.templ — a new sort is one
entry in each. The sort value is a BOUND PARAM compared inside the
cascade, never interpolated; unknown keys resolve to the default.
Filters: series-name matches in WHERE (the group key, so counts stay
true); author (BOOL_OR any-book match) and status (finished/reading/
unstarted vs the viewer's progress, same 0.999 threshold as the
filter_query status arms) in HAVING so COUNT(*) keeps reporting the
whole series. GetDistinctSeriesCount shares the JOIN + HAVING so
pagination totals match. The reading_progress join can't multiply rows
(the table is UNIQUE per media_item+user).
Cards gain the viewer's finished-count ("X of N read" + progress bar),
and the empty state distinguishes "no series at all" from "no matches —
clear filters". The JSON /api/series endpoint takes the same params and
gains read_count/last_read_at additively.
Integration test pins every sort key against a three-series fixture
with distinct created/author/count/progress answers, the filter matrix,
composition, unknown-sort fallback, card read-counts and the JSON
contract — all as the fresh regular user so the live reader's data is
untouched.
Found while double-checking /login after the redirect fix: a wrong
password left the form completely silent. Two stacked causes, both
pre-dating the CSP work:
1. The two HX-Request branches for invalid credentials returned JSON,
unlike every sibling error branch — there was no HTML partial to
show at all. They now return a shared invalidCredentialsHTML div
(the JSON shape is kept for non-htmx API callers).
2. htmx does not swap 4xx/5xx responses by default, so even the HTML
error partials that DID exist (429 lockout, 500s, registration
failures like "Email already exists") never landed in the target.
auth_forms.ts now swaps them itself on htmx:beforeSwap when the
error response is text/html — JSON errors stay ignored.
Verified live: /login success → /dashboard; wrong password shows
"Invalid email/username or password." and stays on /login; duplicate
email on /register shows "Email already exists". Gates: make test 9
pkgs, vitest 155, Playwright 20.
The htmx login/registration partials answered success with an inline
<script> that stashed the auth token and set window.location — refused
by hash-mode script-src (no 'unsafe-inline'), so the user saw "Login
successful! Redirecting..." and nothing else (reported from the home-
page login form; /login was equally broken). The same pattern lived in
collections.go (delete → redirect) and devices.go (token regen →
reload, whose script tag was also unclosed).
Replacement, per the AGENTS.md CSP rules:
- auth.go login/registration: the partial now carries data-auth-token +
data-redirect (redirect is path-only — open-redirect hygiene — and
escaped); web/src/auth_forms.ts (new, armed from main.ts on every
page) listens for htmx:afterSwap, stashes the token, and navigates.
- collections.go delete: data-redirect, same listener.
- devices.go token regen: htmx's native HX-Refresh response header.
- The dead localStorage 'user'/'refreshToken' writes are dropped — no
reader anywhere in web/src; only 'token' is load-bearing (19 modules).
Tripwire extension: internal/middleware/csp_go_sources_test.go scans
non-test Go sources for literal <script> tags and native on*= attributes
(hand-written files only — templ codegen is owned by the .templ scan),
verified to fire on a reintroduced probe. AGENTS.md CSP section notes
the Go-emitted-partial vector.
Verified live: home-page login form → /dashboard with token stashed and
dashboard loading clean; /register form → /dashboard; collections and
devices share the same mechanisms (data-redirect / HX-Refresh). Gates:
make test (9 pkgs incl. both tripwires), vitest 155, Playwright 20,
tsc/vp clean.
Per issue card: a Retry button — error-severity issues previously had no
action at all (only warning/info got Dismiss). Right of the Errors stat
card: Retry all, which builds a queue of every issue currently on the
page and runs them strictly one at a time through the per-issue retry
endpoint. The browser is the queue — resource use is bounded to a single
book in flight, and no batch endpoint exists to maintain.
While the queue runs, the button shows an n/m counter and cards tint via
client-local state classes (the sanctioned island shape; the cards
themselves stay templ-rendered). After the queue drains, the page swaps
in the new /admin/libraries/:id/issues/content templ partial — the
unlinked-books htmx.ajax pattern with a reload fallback — so resolved
issues drop off server-side while failures remain listed, with a
summary toast and per-item error toasts carrying the real failure
reasons.
CSP discipline holds throughout: Alpine @click component methods, no
inline handlers, no js:-prefixed hx-vals.
POST /api/libraries/:id/scan accepts an optional {"folder_path": "..."}:
the scan then covers exactly that one registered folder's subtree instead
of the whole library. The path is validated against the library's
registered folders — anything else is a 400, never trusted. An omitted
body keeps today's full-library behavior, and coalescing (409 with the
active job) is unchanged.
The LibraryPanel folder rows get a per-folder Scan button: pure htmx
attributes posting to the same scan endpoint, with hxValsFolderScan
emitting plain eval-free JSON for hx-vals alongside the existing
hxValsFolderDelete precedent. Progress surfaces through the existing
active-scan polling; no new client code.
Safety: the scanner's archive-reconciliation guard (previous commit)
skips partially-covered libraries, so a one-folder scan cannot mark the
library's other folders' items missing.
POST /api/libraries/:id/issues/:issueId/retry (admin, family-lens gated
via the existing gateIssue) re-runs the work that produced an open issue
— deliberately NOT a library-wide rescan. Two shapes mirror the two
ledger entry kinds:
- item-backed issues rescan their media item through the book-detail
RescanMediaItem mechanism (self-resolving folders, multi-file
audiobook routing, write-after-success guards) and then close the
issue (RetryMediaItemIssue);
- row-less import_failed issues re-import the single file through the
watcher's standalone processMediaFile path (RetryFailedImport: folders
resolved RescanMediaItem-style so paths stay library-root-relative),
whose create-path self-heal plus an explicit path resolve closes the
issue even when the content was already imported by a prior attempt.
Failures return the REAL underlying error so the UI can show why the
issue still exists; a vanished file is a 409 (ErrRetryFileMissing);
already-resolved issues reject re-retry.
Docs + bruno mirrors regenerated for the new route (the generator's seq
renumbering touches three unrelated bruno files as fallout; openapi
gains the path and the date bump), and the docs schema registry gains
the retry entry so TestDocsMatchRoutes passes.
Two scanner-integrity changes supporting the Processing Issues retry work:
- The size-change and sidecar-change update paths only printf'd their
failures, so a book whose metadata update kept failing (the Crusader
genre-22001 loop) degraded silently on every scan with nothing on the
Processing Issues page. Both paths now record an item-backed
metadata_extraction_failed issue — the same type and shape the
sidecar-extraction guard already used — and resolveIssuesForItem
clears it when an update finally succeeds.
- The archive reconciliation pass compares every library item against
what one scan saw on disk. The new per-folder scan scope runs a scan
covering a single folder, which would mark every item under the
library's OTHER folders missing and archive them on a later pass.
The pass now skips a library whose registered folders the scan only
partially covered (GetLibraryFolders vs the scan's folder set),
logging the skip.
applyAudioTagFallbacks captured the ffprobe genre atom verbatim into
media_items.genre — no splitting, no shelf resolution, no cap. That raw
capture is what fed 100+-char Audible category blobs into the column
(the 2026-10 Crusader import/update failures).
The ABS sidecar path already has the correct doctrine (ab2a5568): a known
lexicon shelf wins the genre column, everything else is tags-only. Give
the tag fallback the same treatment — splitGenreCandidates +
pickSmartGenre for the column, split candidates appended to tags
(NormalizeTags dedups). Demoted genres like "Audiobook" stop masquerading
as a shelf, and an unknown blob can no longer manufacture an overflowing
value at all.
Fixtures pin the new behavior: the real 111-char Crusader blob stays out
of the column entirely and lands as six separate tags; a shelf-carrying
blob still wins the column; the previously-pinned verbatim "Audiobook"
expectation is updated to the demoted (tags-only) contract.
Unidentified sync files used to be stuck: every KOReader re-sync
re-queues them, and there was no way to say "this isn't a book" or "I
don't want this one on the server".
Ignore is the durable dismiss: a new ignored_at column hides the row
from the page list, dashboard badge, and per-device API while the sync
keeps refreshing its metadata (the upsert never clears the column).
A collapsible "Ignored" section lists dismissed rows with Restore and
Delete so a misclick is recoverable. Delete is the hard exit — the row
is gone, and honestly re-queued if the device syncs that file again
(the confirm dialog says so).
Actions are page-scoped SSR routes: htmx swaps in the re-rendered
section so counts and buckets can never drift from the DB; ownership
is row → device → user (admins may act on any row). Integration tests
pin the lifecycle (hide everywhere, survive the sync upsert, restore,
delete), foreign-user 403s, and the render surface.
New users column surfaced as an On/Off dropdown in the profile form.
The form and API carry "true"/"false" strings with the theme-style
""-guard, so JSON clients that omit the field can't clobber it;
anything else is rejected. GET/PUT /api/auth/profile expose
keep_screen_awake alongside the other profile fields, and the reader
route threads the preference into its template metadata.
Integration tests pin the schema default, both values, omission
semantics, and rejection — dedicated test user, never testuser.
The combined profile handler (self-edit PUT /api/auth/profile and
admin PUT /api/auth/profile/:id) binds one struct for both paths —
and it's the admin-named one. Its twin UpdateProfileRequest had zero
references left. Delete it and rename AdminUpdateUserRequest to
ProfileUpdateRequest so the name matches who actually binds it; the
comments describing field semantics now stand on their own instead of
pointing at the deleted twin. Wire format (json/form tags) unchanged;
docs schema registration follows the rename.
docTitleOverrides now pins the label with lowercase 'and' (the naive
titleCase fallback capitalized every word); only the sidebar label was
affected — the page h1 comes from the markdown heading and was already
correct. Test pin added to TestDocTitle.
Follows the developer/api → API Reference carve-out: placement-only, no
file paths or URLs change. placeDoc routes user/devices/* to a new
top-level "Devices" section (icon: device) and user/admin-* to a new
top-level "Administration" section (icon: shield); User Guide keeps its
direct guides with no groups. Sidebar order: Getting Started, User
Guide, Devices, Administration, API Reference, Developer, Operations,
Contributing.
Verified live: on kobo-setup the Devices section opens with the three
device guides; on admin-guide Administration opens with both admin
guides; on user/dashboard User Guide opens as before. Tests
(TestPlaceDoc, TestBuildNavigation) updated to pin the new placement.
script-src now allow-lists the guard by sha256 of its rendered bytes
(middleware.ScriptHashThemeFlashGuard) instead of 'unsafe-inline' — the
guard is the app's only inline <script>. security_headers_test.go renders
templates.ThemeFlashGuard() and fails with the replacement hash if the
script text ever drifts.
Executing anti-XSS strict CSP surfaced three latent violations the
eval-removal sweep couldn't see (hash mode is stricter than eval mode,
because inline event handler ATTRIBUTES now count against script-src):
1. admin_archived / admin_library: 15 native onclick=/onsubmit= handlers
(modal open/close, folder-browser picker, purge button) converted to
Alpine @click/@submit component methods — openCreateLibraryModal,
hideModal, pickBrowsePath added to the admin component;
purgeArchivedItems promoted from its window global into the component
and the (window as any) exposure removed.
2. dashboard/docs: onclick handlers converted — dashboard's
event.stopPropagation() → @click.stop; docs' sidebar/search closers →
closeDocsSidebar/closeSearchResults component methods.
3. 8 onerror= image-fallback attributes (cover placeholders app-wide +
audiobook cover hide) replaced by data-fallback attributes handled by
one delegated capture-phase listener in web/src/image_fallback.ts —
works for htmx-swapped content without re-binding.
Verified: theme guard applies a saved theme on public pages under the
hash CSP; 14-page authenticated sweep clean; admin pages clean under the
promoted admin; admin-library modal open/close bindings exercised;
readers render error-free; go build/vet/middleware-test, tsc, vp
check/test (146), build:ts all green.
The strict CSP sweep surfaced a final class of bindings on the audio
reader and book detail pages that referenced browser globals
(Number, String, Math, window, document) — identifiers the CSP-safe
evaluator resolves only from component scope:
- audio reader: scrub input, volume slider/input, speed+skip active
styles, genre tick styles, and the default-speed select now route
through component methods (onScrubInput, setVolumeFromEvent,
applyDefaultSpeedFromSelect, volumeLabel, railTickStyle,
isSkipActive, isSpeedActive, isDefaultSpeed)
- book detail: cover-upload click and banner edit validation moved to
component methods (triggerCoverUpload, bannerEditHasText)
- admin: Refresh-to-View-Books reload moved to reloadPage()
- reader: note edit validation moved to noteEditHasText()
CSP header: frame-src 'self' blob: (foliate renders book content in
a blob: frame) and blob: on font-src/img-src (EPUB-embedded fonts
and images load as blob: URLs inside that frame). 'unsafe-eval' is
gone: script-src is 'self' plus 'unsafe-inline' for the pre-paint
theme flash guard.
Also: header openPanel promoted to the root component (one open
panel across all sidebar panels, persisted in init), with the
sidebar shell reduced to themeMode state + methods; fmtTime restored
to the audioReader component (ripped by the unexport pass — time
displays were dead on the audio reader).
- audio reader: scrub/volume/speed/genre-tick bindings converted to
component methods (onScrubInput, setVolumeFromEvent,
applyDefaultSpeedFromSelect, volumeLabel, railTickStyle,
isSkipActive/isSpeedActive) — these used Number/Math/String globals
that the CSP evaluator cannot resolve
- fmtTime restored to the audioReader component: the unexport pass
had ripped it from the Alpine.data object (time displays were dead)
- book detail: cover-upload click and banner edit validation moved to
component methods (triggerCoverUpload, bannerEditHasText)
- admin/docs/index components: init() hooks for their x-init chains
- CSP header: 'unsafe-eval' dropped — the CSP build needs no eval
- e2e spec: replace test.info().annotate (unavailable in this
Playwright version) with console logging
The metadata editor's Comic/Manga section (Manga Type, Reading
Direction, Age Rating, Story Arc, Imprint, Scan Info, Metadata
Notes, B&W) rendered for every item — an audiobook showed a manga
dropdown, which is nonsense.
MediaDetail now carries LibraryTypeName (resolved via
GetLibraryType in the /media/:uuid handler) and a
ShowComicMetadata() gate: comics and manga libraries always show the
section; other media only when the scanner recorded comic-style
values (fixed-layout manga-style EPUBs get those scanner defaults
deliberately — when present they stay editable).
Audiobookshelf-style exports join genres into one comma-string
(["Epic, Action & Adventure, Military, Sword & Sorcery"]). The
smart-genre picker already split candidates for the genre column,
but the tags assembly used the raw unsplit array — so the whole
string landed as ONE element of media_items.tags and rendered as a
single tag everywhere.
Use pickSmartGenre's split candidates for tags too, exactly as the
function's own comment documents ('split parts included stay in
tags'). Commas inside a single subject heading remain legal within
one tag (EPUB dc:subject path unchanged).
Two regressions from the cleanup cycle, caught in E2E:
1. The unexport pass stripped loadTheme, loadUserTheme, and
updateThemeIndicators from the Alpine.data("theme") registration —
the member-removal pass could not tell export-block lines from
Alpine component object lines. Theme dropdowns on docs and
non-app pages stopped reflecting the active theme. Members
restored (functions were kept, only their scope registration was
lost).
2. The new Content-Security-Policy shipped script-src without
'unsafe-eval'. Alpine evaluates every @click / x-init / x-show
expression through new Function(), so the CSP silently disabled
ALL Alpine interactivity app-wide: sidebar categories stuck open,
modals rendering unhidden and unresponsive, dropdowns inert.
script-src now includes 'unsafe-eval' with a comment explaining
the Alpine requirement.
Lifecycle (internal/app + main.go):
- App.OnShutdown hooks run after the HTTP server drains, in
registration order: Worker.Shutdown (background jobs now drain
instead of dying on SIGTERM), DeviceAuthMiddleware.Stop (releases
the rate-limiter ticker), Handler.CloseScanner (fsnotify watcher
and scan log files)
Security headers:
- SecurityHeadersMiddleware (survivor of middleware/security.go,
renamed security_headers.go) is now on the echo stack: nosniff,
frame-deny, HSTS, CSP (unsafe-inline required by the sanctioned
theme flash guard + Alpine), referrer and permissions policy
Scanner settings cache:
- settings saves (UpdateScanSettings + ApplySetting for the three
scan keys) now invalidate MediaScanner's 30s settings cache via
SystemSettingsHandler.SetScanSettingsChanged, so changes apply on
the next poll instead of after the TTL
- MediaScanner.Close closes the scan logger, fixing a file-descriptor
leak on every per-item rescan (fresh scanners never closed logs)
Client fix:
- custom-section-builder's add/remove-selection buttons used inline
onclick attributes referencing module-scoped functions, which are
not global in a bundled ES module: the buttons threw ReferenceError.
Functions restored and exposed on window via declare global
Also fixes a pre-existing vet error (saveCoverImage copied
echo.Context by value, mutex included) and drops the per-key
SettingsCache.InvalidateKey variant left callerless by the
full-flush invalidation wiring.
Repo-wide gofmt sweep: const-block alignment, doc-comment spacing,
statement-after-brace fix in media.go, trailing blank lines. No
semantic changes — whitespace-insensitive diff is empty for every
file in this commit.
Root cause of 'clicked from overview, bold but menus closed': cross-page
markdown links point at area/file.md; the handler happily loads .md
URLs, but the sidebar keys on the URL path — contains-URL expansion and
the active-link styling compare against extension-less nav URLs, so a
.md path rendered the page with its menu closed and the current item
unhighlighted. (My curl checks had used clean URLs, which is why it
worked in testing but not from the overview page.)
Fix: ShowDocumentation 301-redirects .md paths to the clean canonical
form (query preserved; browsers re-attach fragments), and
renderDocPage normalizes the path for nav expansion/grouping as a belt-
and-braces. Live-verified: .../get_ratings.md -> 301 ->
clean page with both containing menus open and the get_ratings link
carrying the active class.
Simplification per spec: the menu tracks state through the URL alone.
Land on /docs: fully closed, headers only (an explicit home rule now
overrides the contains-URL rule, since the root overview doc lives in
Getting Started and would otherwise auto-open it). Land on any doc page:
its containing section + group render open at first paint, current link
bold — no JS, no jump, arrows consistent. Navigate elsewhere: old
closes, new opens.
Deleted: docs_nav cookie const/parse, ApplyNavOverrides, cookie write +
collect/persist helpers in web/src/docs.ts, data-key attributes in the
sidebar templ, and the first-section-open fallback (docs home used to
force-open the first section — spec'd as closed).
toggleCollapsible remains for in-page peeking; purely ephemeral.
The [data-collapsed=true] CSS from the previous round stays — it is what
makes the server-rendered state actually hide sections at paint (the
original bug was that this CSS never existed).
Tests: TestBuildNavigationNothingCurrent now asserts zero sections open
on unmatched paths; TestBuildNavigation asserts Getting Started closed
on a device page (no site-wide pin). make test green; 146 vitests; live
verified: home 0 expanded, doc pages exactly 2 (section+group).
Three problems, one root cause each:
1. Sidebar jump: the server rendered its own default (current section
open, rest collapsed) and restoreDocsNav corrected it after paint —
open-then-close. Fix: persistence moved server-side. toggleCollapsible
now writes a docs_nav cookie (strictly-necessary UI state, no
identifiers, SameSite=Lax); ApplyNavOverrides merges it into the nav
before render, so first paint IS the visitor's state. The post-paint
restore is deleted.
2. Arrows out of sync / double-click: two visibility mechanisms
disagreed. There was NO data-collapsed CSS at all — collapsed sections
were only ever hidden by JS after paint, and the arrow rotation came
from a different (server) state. Now: style.css hides on
[data-collapsed=true] (single source of truth, server-rendered),
applyCollapsed sets only the attribute, arrows rotate from the same
merged value. Live check: 37/37 collapsibles arrow-consistent.
3. api-reference.md links: all 20+ quick links were directory-style
(authentication/ etc.) that 404 on the docs site, plus stale targets
(books/, ../collections-api.md, websocket 'coming soon'). Rewritten to
real per-file links — all 9 unique targets return 200 live. Same rot
fixed in admin/hash-conflicts.md, development.md, android-app.md,
create_media_item.md, koreader-protocol.md; docs/index.md redirected
from the deleted collections-api monolith (stale 530-line duplicate of
the gated api/collections docs — deleted, per the WO2 doctrine).
New gate: TestDocsRelativeLinks walks docs/developer/** and fails make
test on any relative link that is directory-style or unresolvable —
overview links can never silently rot again.
Theme flash (admin pages, docs, stats, progress — anywhere SSR'd):
the JWT middlewares built the context user WITHOUT the Theme column,
so every page rendering theme-{user.Theme} from c.Get("user") painted
an empty class (:root tokyo-night default) and only settled on the real
theme after initializeTheme -> loadUserTheme round-tripped the server.
Both middlewares now carry liveUser.Theme into the context user;
verified live: /admin /stats /progress /docs /docs-deep /dashboard
/collections all SSR theme-hackerman.
Defense in depth for the same bug: loadTheme() no longer re-applies
localStorage over a server-rendered body theme (applyTheme records even
the default, so any signed-out visit baked tokyo-night into localStorage
and every signed-in page flipped to it post-paint). Public auth/landing
bodies dropped their hardcoded theme classes so ThemeFlashGuard's
pre-paint html class actually governs them; docs theme <select> now
syncs from the body class and re-syncs after the profile fetch.
Sidebar state (the unpleasant part): the app sidebar's openPanel was
reinitialized from the pathname on every load; it now restores from
localStorage ('sidebar-open-panel') with the pathname default as
fallback and -persists every toggle. Docs sidebar sections/groups
carry data-key attributes and toggleCollapsible persists collapsed
state per key ('docs-nav-state'); restoreDocsNav() applies it at init —
server-rendered defaults still govern first visit.
make test green; rendering ownership OK (no new inline scripts); 146
vitests pass; live-verified SSR classes on 7 pages.
Three follow-ups from the smoke run, all additive:
1. Status asserts: bruno_gen.go parses each doc's documented success
status (## Response (2NN), 'Success is **201**', '**204 No Content**')
and emits a runtime tests script (expect(res.status).to.eql(N)) on
148 of 191 mirrors. GETs default to 200 (uniform across this API);
mutating verbs stay assert-less unless documented — no guessed
asserts. Verified live: asserts execute and fail loudly (deliberate
999 probe), pass on real responses.
2. OpenAPI: the same -update run now also writes
docs/developer/openapi.yaml (141 paths, tags from docs areas,
security schemes per auth label, request examples literal, no
invented schemas — the docs site remains the shape reference).
Verify mode fails when the file is missing.
3. Scenarios: 7 safe negative scenarios rebuilt (auth registration
matrix 409/403/400, invalid-uuid 400, unknown 404 x2, invalid range
400) — all no-mutation, repeatable, run via bru: 7/7 tests pass.
These replace a fraction of the deleted broken suite; mutations-
heavy scenarios stay out until a design for setup chains exists.
created_at RFC3339 deliberately NOT done — it is the one item that
would break things (Android parses current formats; plan reserves it
as a compat decision).bru run of mutating mirrors also proven to touch
live data (a 'To Read' collection was created and deleted in testing);
only id-less GET mirrors and negative scenarios are safe for blanket
runs.
make test green; generation idempotent (byte-identical re-runs);
no stray smoke rows in users.
Curated smoke run of the generated collection against the live dev
server (37/37 after fixes): registration matrix (409 duplicate email,
403 admin-creation-with-admins-present, 400 weak password), non-admin
403s, device-token/opds 401s, collection CRUD incl. the documented 201
update quirk and 204 delete, stats routes, scanner, jobs, malformed-uuid
400s. Testuser untouched (password test uses a guaranteed-wrong current
password -> 401); the smoke user was admin-deleted in cleanup and its
token verified dead.
Correction the smoke run forced: the batch-8 'password_hash leak' was a
false alarm — my registry pin ([]database.Users) never matched the
handler, which returns []database.ListUsersRow (sqlc join with
device_count). Live response confirmed: id/email/username/theme/names/
role/max_devices/created_at/updated_at/device_count — NO hash. Doc
example, field table, and registry pin all corrected; leak note
replaced with the live-verified statement.
Also verified via the real Bruno CLI: 9 safe generated requests
(4 shown passing incl. stats/collections/devices; password-rules
public) execute against the server with collection-level auth.
Full-collection bru run deliberately not attempted (mutations in
static bodies would touch live data).
The generator moves into the Go toolchain per repo preference:
internal/docs/bruno_gen.go emits the OpenCollection mirrors (same output
contract — gate-verified example-request bodies, env-var params,
device-token headers, login/register/refresh token-capture scripts) and
TestBrunoMirrorsDocs grows a -update flag: verification by default,
regeneration on demand:
go test ./internal/docs -run TestBrunoMirrorsDocs -update
Generator and checker are now the same code — they cannot drift apart.
scripts/gen_bruno_from_docs.py deleted; opencollection.yml README points
at the go-test command.
Also fixed vs the Python pass: mirror placement follows the docs tree
folder (RelPath), not the area frontmatter (several docs — highlights,
notes, ratings, users, progress — carry a different area); docs blocks
emit the collection's mapping form (content + type), not a bare scalar;
JSON body sanity is checked at generation time.
Idempotent: back-to-back -update runs are byte-identical.
make test green (9 pkgs).
WO4. The old collection was the drifted artifact AGENTS.md warned about:
bodies pre-truncated at their first {{var}}, routes renamed server-side
but not here (/api/notes/:media_id, /api/book-matching/search,
/api/collections/:id/rules/test), an orphan universal-progress suite for
routes that no longer exist.
scripts/gen_bruno_from_docs.py emits one OpenCollection request per
frontmatter'd endpoint doc (191 requests): gate-verified example-request
JSON as body, {{env}} params per the environment file, device-token
routes get an explicit Bearer {{device_token}} header, login/register/
refresh keep the token-capture after-response scripts that chain the
collection. Old 1:1 mirrors deleted (128); hand-authored NewDevDBSetup,
environments, and edge-case suites kept. 30 scenario files with
truncated bodies (the documented corruption) deleted — they could not
run.
internal/docs/drift_test.go gains TestBrunoMirrorsDocs: every
frontmatter'd doc must have a Bruno request with matching method+URL
shape — the collection can never silently drift from the docs again.
opencollection.yml README block replaced with the real run instructions.
make test green (9 pkgs, parity test included in internal/docs).
Stats docs written handler/service-first (summary/this-week/book-stats
new; library-usage frontmatter'd — its body was already accurate).
Registry: 4 entries on services.UserSummaryStats/WeekSummaryStats/
LibraryUsageStats/BookStats. Family-lens note: library-usage 403s an
explicit library_id outside the visible set (431fec0 rule).
UNVERIFIED sweep (7 -> 0), each resolved against the handler:
- device-usage takes NO date params; popular-books takes only ?limit
(the 'standard analytics date-range params' claim was wrong)
- link_book media_item_id: no validate tag but service-always-written
- push_stats book-ref field set documented (uuid/sha256/title/authors)
- filter_count: confirmed ParseMediaSearchParams machinery + 400 on bad
filter + 403 visibility
- facets values element shape pinned: {value, display?, count}
- get_config: dynamic key set IS the contract (reworded, no marker)
make test 9/9.
All remaining endpoint docs frontmatter'd: media-item bulk ops + audio
surface, collections book-listing/preview, koreader rating/review/stats
plugin protocol, hash-conflicts JSON list.
Scope call (Claude): resolve_hash_conflict.md DELETED — the resolve POST
renders an htmx fragment (renderResolved), a web-admin flow under rule 3;
only the GET list is a JSON API. update_media_review skipped by Beam
(correct): ratings/upsert_media_review.md already covers the PUT.
Registry: 16 entries. koreader rating/review/stats all bind
koreaderBookRef (query or body) — odd=half-star rating scale; audio track
endpoint serves raw bytes (ServeContent, Range) — response nil. Fixed
drift: deleted_annotations doc's secondary restore/purge fences
unlabeled (gate validates labeled blocks against the ONE frontmatter
route); chapters example labeled. UNVERIFIED: 1 (koreader book-ref
fields, push_stats). make test 9/9.
WO3 sync internals resolved: the book-matching docs ARE the sync surface
(books/query, link-book, bulk/auto-link, unlinked-books, suggestions,
file-alias CRUD) + collections rule/test/bulk-assign/shelf-mapping
surface.
Dedupe resolved (Claude call): search_books.md declared
GET /api/books/match but its BODY documents POST /api/sync/books/query —
both routes exist and are distinct (query-param device protocol vs JSON
body plugin protocol). Repointed search_books.md frontmatter to the POST
route; get_book_matches.md keeps the GET.
Claude registry: 15 entries (one pre-existing link-book entry deduped).
Drift fixed by gate: auto-link request {confidence_threshold, limit} not
{device_id, threshold}; bulk-link link objects {unlinked_book_id,
media_item_id, confidence_score} not {device_id, book_id, ...};
responses {results,total,successful,failed}/{auto_linked,results}; added
missing example-responses (bare delete, remove-rule is_smart-after
semantics, bulk-assign per-book results). make test 9/9.
Highlights + notes CRUD (media-item scoped, jwt), progress delete,
reading-stats analytics, and the OPDS trio frontmatter'd. OPDS docs are
the first XML entries: example-response intentionally omitted (Atom
responses), gate contract honored via nil-Response registry pins.
Gate fix: group .Use() middleware now parsed (opds.Use(DeviceAuth...
Authenticate) previously resolved to auth 'none'). Registry: 13 entries.
Drift fixed by gate: reading-stats key names (total_reading_time_minutes,
most_active_day_of_week, daily_reading_minutes — draft invented
reading_time_hours/books_completed); notes/highlights list endpoints
return BARE arrays (no {notes:[]}/{highlights:[]} wrapper); create-note
request has no percentage_location/epubcfi_location. New
/tmp/merge_arrays.py handles array-example merges.
make test 9/9.
14 endpoints backfilled: devices pending/approve/reject; users
password/profile/profile-delete self-service (admin variants in prose);
admin list_users + update_user_max_devices; progress history; book
matching; new password-rules doc.
Gate fix: authLabel missed JWT middleware written inline as
createJWTMiddleware(cfg) (uppercase 'JWT') — reported auth 'none' for
/api/auth self-service routes and would have for any wrapper-inlined
group. Labeled 'JWTMiddleware' + fallthrough 'jwt' case.
Claude registry: 12 entries. Drift fixed by gate: list_users on-device —
handler returns raw []database.Users (bare array, full model incl.
password_hash — real leak the API emits; pinned + documented; earlier
draft had invented {users,total,limit,offset} wrapper). max-devices
response is {message} only. books/match is GET /api/books/match
{matches, action}. progress history sessions = ReadingHistory rows
(pgtype fields). make test 9/9.
Scanner surface frontmatter'd (scan/start/stop/cancel/status/active/
active-summary + watch start/stop/status) with provenance comments;
active-summary on a non-admin group.
Claude registry: 10 entries. Fixed handler drift Beam missed: scan_library
request has no "recursive"; scanner/start binds folder_paths (not
invented interval_seconds); scanner/stop binds no body; response is
{message, job_id, status} with 409 coalescing variant
{error, active_job_id}.
Gate fix in drift_test seed: registerScannerRoutes group seeds used bare
"jwt"/"admin" tokens invisible to authLabel; and silently passed before
only because the seeded set was unchecked. Seeds now 'jwtMiddleware'/
'AdminMiddleware' markers. Also: cleanup of a stale-merge-script residue
in collections/update_rule.md example (extra null keys). make test 9/9.
Kobo emulation + KOReader calibre-compatible surface backfilled with
device-token auth; collections reorder + update_rule created.
Claude registry: 12 entries. kobo/koreader emulation endpoints all bind
arbitrary protocol POST bodies (markup/bookmark/gettests/sync-from-server,
koreader progress/bookmarks) — registered as AnyJSONObject, not bodyless as
first drafted. Initialization returns the library sync payload, not the init
response.
Contract correction in update_rule.md: the route is SetRuleEnabled ({enabled}
required -> {collection_id, is_smart}); Beam's draft invented rule-field
updates. Reordered response/status: {item_ids} -> {status: reordered}.
make test 9/9.
Beam backfilled 11 library docs (CRUD, folders, visibility, browse, stats,
media-items) and re-attached frontmatter to create_collection.md — which had
silently lost its pilot version in cc893f75: the negative-test
'git checkout --' restored the OLD file because the pilot was never staged,
and a frontmatter-less doc is invisible to the gate. Claude restored the full
corrected pilot content (priority higher-wins, time.String() note, 409).
Claude registry: 11 entries + one inverted catch — DELETE /:id/folders DOES
bind {folder_path} (handler reuses AddLibraryFolderRequest); registry first
draft said bodyless. GET /:id/visibility returns joined
GetUserVisibleLibrariesRow (type_name etc.), not bare Libraries.
Gate-driven merges fixed create/get library response shapes (sqlc row keys:
created_by_admin_id, library_type_id, updated_at; dropped invented type/
folders/is_visible). Fixed invented request fields (user_id in
set-visibility; library_type_id in update). make test 9/9.
Beam backfilled list/get/create/update/delete media-item, search, ratings
CRUD, progress GET/PUT. Beam correctly identified create/update/delete as
jwt+admin (admin group in the router — verified).
Claude registry: 12 entries. Notable pins:
- get_media_item: hand-built ~44-key map pinned verbatim from the handler
- create/update return the full database.MediaItems row (74 columns) —
examples merged up to the real shape via gate-driven merge script
- search returns []SearchMediaItemsRow (sqlc row — Real type pinned)
- rating scale is 1-10 int (odd = half-star); GET returns {rating: null} when
absent; update reuses the create handler (201)
- progress GET: 25 base keys + 4 CONDITIONAL position-authority keys
(css_selector/anchor_href/char_offset/context_text — added by the handler
for convertible formats with a valid CFI); PUT progress binds
sync.SaveProgressRequest and returns database.ReadingProgress
- Beam's examples were missing entirely on 4 files (list/search/progress
GET/PUT) — relabeled existing fences or inserted contract-shaped blocks
Gate-driven merge scripts (object + array variants) brought all examples to
the exact handler key sets. make test 9/9.
Beam backfilled auth core (login/register/refresh/logout/profile) and devices
core (list/update/delete/shelf CRUD/sidecar/register flow) with frontmatter,
provenance, and labeled examples. All claimed shapes verified against
handlers: AuthResponse {access_token, refresh_token omitempty, token_type,
expires_in, user}, update device {device_updated, device}, shelf wrappers,
DeviceAuthStatusResponse.
Claude registry verification: 15 entries pinned by type (LoginRequest,
RegisterRequest, AuthResponse, RefreshTokenRequest/Response, UserProfile,
DeviceListResponse, DeviceUpdateRequest, AddToShelfRequest, SidecarConfig,
DeviceRegistrationRequest/Response, DeviceAuthStatusRequest/Response).
Removed revoke_device.md — name-only duplicate of delete_device.md (same
DELETE /api/devices/:id route; was the last audit-orphan doc).
Gate catches this round (subtle Go serialization facts, now pinned in docs):
- uuid.UUID arrays never serialize as omitted: register/status response
ALWAYS carries device_id (all-zeros while pending) despite omitempty
- update device request binds device_name, not name
- register response includes device_id alongside registration_id
- list devices response is a {devices, total} wrapper
John O'Keefe