Root cause of 'clicked from overview, bold but menus closed': cross-page
markdown links point at area/file.md; the handler happily loads .md
URLs, but the sidebar keys on the URL path — contains-URL expansion and
the active-link styling compare against extension-less nav URLs, so a
.md path rendered the page with its menu closed and the current item
unhighlighted. (My curl checks had used clean URLs, which is why it
worked in testing but not from the overview page.)
Fix: ShowDocumentation 301-redirects .md paths to the clean canonical
form (query preserved; browsers re-attach fragments), and
renderDocPage normalizes the path for nav expansion/grouping as a belt-
and-braces. Live-verified: .../get_ratings.md -> 301 ->
clean page with both containing menus open and the get_ratings link
carrying the active class.
Simplification per spec: the menu tracks state through the URL alone.
Land on /docs: fully closed, headers only (an explicit home rule now
overrides the contains-URL rule, since the root overview doc lives in
Getting Started and would otherwise auto-open it). Land on any doc page:
its containing section + group render open at first paint, current link
bold — no JS, no jump, arrows consistent. Navigate elsewhere: old
closes, new opens.
Deleted: docs_nav cookie const/parse, ApplyNavOverrides, cookie write +
collect/persist helpers in web/src/docs.ts, data-key attributes in the
sidebar templ, and the first-section-open fallback (docs home used to
force-open the first section — spec'd as closed).
toggleCollapsible remains for in-page peeking; purely ephemeral.
The [data-collapsed=true] CSS from the previous round stays — it is what
makes the server-rendered state actually hide sections at paint (the
original bug was that this CSS never existed).
Tests: TestBuildNavigationNothingCurrent now asserts zero sections open
on unmatched paths; TestBuildNavigation asserts Getting Started closed
on a device page (no site-wide pin). make test green; 146 vitests; live
verified: home 0 expanded, doc pages exactly 2 (section+group).
Three problems, one root cause each:
1. Sidebar jump: the server rendered its own default (current section
open, rest collapsed) and restoreDocsNav corrected it after paint —
open-then-close. Fix: persistence moved server-side. toggleCollapsible
now writes a docs_nav cookie (strictly-necessary UI state, no
identifiers, SameSite=Lax); ApplyNavOverrides merges it into the nav
before render, so first paint IS the visitor's state. The post-paint
restore is deleted.
2. Arrows out of sync / double-click: two visibility mechanisms
disagreed. There was NO data-collapsed CSS at all — collapsed sections
were only ever hidden by JS after paint, and the arrow rotation came
from a different (server) state. Now: style.css hides on
[data-collapsed=true] (single source of truth, server-rendered),
applyCollapsed sets only the attribute, arrows rotate from the same
merged value. Live check: 37/37 collapsibles arrow-consistent.
3. api-reference.md links: all 20+ quick links were directory-style
(authentication/ etc.) that 404 on the docs site, plus stale targets
(books/, ../collections-api.md, websocket 'coming soon'). Rewritten to
real per-file links — all 9 unique targets return 200 live. Same rot
fixed in admin/hash-conflicts.md, development.md, android-app.md,
create_media_item.md, koreader-protocol.md; docs/index.md redirected
from the deleted collections-api monolith (stale 530-line duplicate of
the gated api/collections docs — deleted, per the WO2 doctrine).
New gate: TestDocsRelativeLinks walks docs/developer/** and fails make
test on any relative link that is directory-style or unresolvable —
overview links can never silently rot again.
Theme flash (admin pages, docs, stats, progress — anywhere SSR'd):
the JWT middlewares built the context user WITHOUT the Theme column,
so every page rendering theme-{user.Theme} from c.Get("user") painted
an empty class (:root tokyo-night default) and only settled on the real
theme after initializeTheme -> loadUserTheme round-tripped the server.
Both middlewares now carry liveUser.Theme into the context user;
verified live: /admin /stats /progress /docs /docs-deep /dashboard
/collections all SSR theme-hackerman.
Defense in depth for the same bug: loadTheme() no longer re-applies
localStorage over a server-rendered body theme (applyTheme records even
the default, so any signed-out visit baked tokyo-night into localStorage
and every signed-in page flipped to it post-paint). Public auth/landing
bodies dropped their hardcoded theme classes so ThemeFlashGuard's
pre-paint html class actually governs them; docs theme <select> now
syncs from the body class and re-syncs after the profile fetch.
Sidebar state (the unpleasant part): the app sidebar's openPanel was
reinitialized from the pathname on every load; it now restores from
localStorage ('sidebar-open-panel') with the pathname default as
fallback and -persists every toggle. Docs sidebar sections/groups
carry data-key attributes and toggleCollapsible persists collapsed
state per key ('docs-nav-state'); restoreDocsNav() applies it at init —
server-rendered defaults still govern first visit.
make test green; rendering ownership OK (no new inline scripts); 146
vitests pass; live-verified SSR classes on 7 pages.
Three follow-ups from the smoke run, all additive:
1. Status asserts: bruno_gen.go parses each doc's documented success
status (## Response (2NN), 'Success is **201**', '**204 No Content**')
and emits a runtime tests script (expect(res.status).to.eql(N)) on
148 of 191 mirrors. GETs default to 200 (uniform across this API);
mutating verbs stay assert-less unless documented — no guessed
asserts. Verified live: asserts execute and fail loudly (deliberate
999 probe), pass on real responses.
2. OpenAPI: the same -update run now also writes
docs/developer/openapi.yaml (141 paths, tags from docs areas,
security schemes per auth label, request examples literal, no
invented schemas — the docs site remains the shape reference).
Verify mode fails when the file is missing.
3. Scenarios: 7 safe negative scenarios rebuilt (auth registration
matrix 409/403/400, invalid-uuid 400, unknown 404 x2, invalid range
400) — all no-mutation, repeatable, run via bru: 7/7 tests pass.
These replace a fraction of the deleted broken suite; mutations-
heavy scenarios stay out until a design for setup chains exists.
created_at RFC3339 deliberately NOT done — it is the one item that
would break things (Android parses current formats; plan reserves it
as a compat decision).bru run of mutating mirrors also proven to touch
live data (a 'To Read' collection was created and deleted in testing);
only id-less GET mirrors and negative scenarios are safe for blanket
runs.
make test green; generation idempotent (byte-identical re-runs);
no stray smoke rows in users.
Curated smoke run of the generated collection against the live dev
server (37/37 after fixes): registration matrix (409 duplicate email,
403 admin-creation-with-admins-present, 400 weak password), non-admin
403s, device-token/opds 401s, collection CRUD incl. the documented 201
update quirk and 204 delete, stats routes, scanner, jobs, malformed-uuid
400s. Testuser untouched (password test uses a guaranteed-wrong current
password -> 401); the smoke user was admin-deleted in cleanup and its
token verified dead.
Correction the smoke run forced: the batch-8 'password_hash leak' was a
false alarm — my registry pin ([]database.Users) never matched the
handler, which returns []database.ListUsersRow (sqlc join with
device_count). Live response confirmed: id/email/username/theme/names/
role/max_devices/created_at/updated_at/device_count — NO hash. Doc
example, field table, and registry pin all corrected; leak note
replaced with the live-verified statement.
Also verified via the real Bruno CLI: 9 safe generated requests
(4 shown passing incl. stats/collections/devices; password-rules
public) execute against the server with collection-level auth.
Full-collection bru run deliberately not attempted (mutations in
static bodies would touch live data).
The generator moves into the Go toolchain per repo preference:
internal/docs/bruno_gen.go emits the OpenCollection mirrors (same output
contract — gate-verified example-request bodies, env-var params,
device-token headers, login/register/refresh token-capture scripts) and
TestBrunoMirrorsDocs grows a -update flag: verification by default,
regeneration on demand:
go test ./internal/docs -run TestBrunoMirrorsDocs -update
Generator and checker are now the same code — they cannot drift apart.
scripts/gen_bruno_from_docs.py deleted; opencollection.yml README points
at the go-test command.
Also fixed vs the Python pass: mirror placement follows the docs tree
folder (RelPath), not the area frontmatter (several docs — highlights,
notes, ratings, users, progress — carry a different area); docs blocks
emit the collection's mapping form (content + type), not a bare scalar;
JSON body sanity is checked at generation time.
Idempotent: back-to-back -update runs are byte-identical.
make test green (9 pkgs).
WO4. The old collection was the drifted artifact AGENTS.md warned about:
bodies pre-truncated at their first {{var}}, routes renamed server-side
but not here (/api/notes/:media_id, /api/book-matching/search,
/api/collections/:id/rules/test), an orphan universal-progress suite for
routes that no longer exist.
scripts/gen_bruno_from_docs.py emits one OpenCollection request per
frontmatter'd endpoint doc (191 requests): gate-verified example-request
JSON as body, {{env}} params per the environment file, device-token
routes get an explicit Bearer {{device_token}} header, login/register/
refresh keep the token-capture after-response scripts that chain the
collection. Old 1:1 mirrors deleted (128); hand-authored NewDevDBSetup,
environments, and edge-case suites kept. 30 scenario files with
truncated bodies (the documented corruption) deleted — they could not
run.
internal/docs/drift_test.go gains TestBrunoMirrorsDocs: every
frontmatter'd doc must have a Bruno request with matching method+URL
shape — the collection can never silently drift from the docs again.
opencollection.yml README block replaced with the real run instructions.
make test green (9 pkgs, parity test included in internal/docs).
Stats docs written handler/service-first (summary/this-week/book-stats
new; library-usage frontmatter'd — its body was already accurate).
Registry: 4 entries on services.UserSummaryStats/WeekSummaryStats/
LibraryUsageStats/BookStats. Family-lens note: library-usage 403s an
explicit library_id outside the visible set (431fec0 rule).
UNVERIFIED sweep (7 -> 0), each resolved against the handler:
- device-usage takes NO date params; popular-books takes only ?limit
(the 'standard analytics date-range params' claim was wrong)
- link_book media_item_id: no validate tag but service-always-written
- push_stats book-ref field set documented (uuid/sha256/title/authors)
- filter_count: confirmed ParseMediaSearchParams machinery + 400 on bad
filter + 403 visibility
- facets values element shape pinned: {value, display?, count}
- get_config: dynamic key set IS the contract (reworded, no marker)
make test 9/9.
All remaining endpoint docs frontmatter'd: media-item bulk ops + audio
surface, collections book-listing/preview, koreader rating/review/stats
plugin protocol, hash-conflicts JSON list.
Scope call (Claude): resolve_hash_conflict.md DELETED — the resolve POST
renders an htmx fragment (renderResolved), a web-admin flow under rule 3;
only the GET list is a JSON API. update_media_review skipped by Beam
(correct): ratings/upsert_media_review.md already covers the PUT.
Registry: 16 entries. koreader rating/review/stats all bind
koreaderBookRef (query or body) — odd=half-star rating scale; audio track
endpoint serves raw bytes (ServeContent, Range) — response nil. Fixed
drift: deleted_annotations doc's secondary restore/purge fences
unlabeled (gate validates labeled blocks against the ONE frontmatter
route); chapters example labeled. UNVERIFIED: 1 (koreader book-ref
fields, push_stats). make test 9/9.
WO3 sync internals resolved: the book-matching docs ARE the sync surface
(books/query, link-book, bulk/auto-link, unlinked-books, suggestions,
file-alias CRUD) + collections rule/test/bulk-assign/shelf-mapping
surface.
Dedupe resolved (Claude call): search_books.md declared
GET /api/books/match but its BODY documents POST /api/sync/books/query —
both routes exist and are distinct (query-param device protocol vs JSON
body plugin protocol). Repointed search_books.md frontmatter to the POST
route; get_book_matches.md keeps the GET.
Claude registry: 15 entries (one pre-existing link-book entry deduped).
Drift fixed by gate: auto-link request {confidence_threshold, limit} not
{device_id, threshold}; bulk-link link objects {unlinked_book_id,
media_item_id, confidence_score} not {device_id, book_id, ...};
responses {results,total,successful,failed}/{auto_linked,results}; added
missing example-responses (bare delete, remove-rule is_smart-after
semantics, bulk-assign per-book results). make test 9/9.
Highlights + notes CRUD (media-item scoped, jwt), progress delete,
reading-stats analytics, and the OPDS trio frontmatter'd. OPDS docs are
the first XML entries: example-response intentionally omitted (Atom
responses), gate contract honored via nil-Response registry pins.
Gate fix: group .Use() middleware now parsed (opds.Use(DeviceAuth...
Authenticate) previously resolved to auth 'none'). Registry: 13 entries.
Drift fixed by gate: reading-stats key names (total_reading_time_minutes,
most_active_day_of_week, daily_reading_minutes — draft invented
reading_time_hours/books_completed); notes/highlights list endpoints
return BARE arrays (no {notes:[]}/{highlights:[]} wrapper); create-note
request has no percentage_location/epubcfi_location. New
/tmp/merge_arrays.py handles array-example merges.
make test 9/9.
14 endpoints backfilled: devices pending/approve/reject; users
password/profile/profile-delete self-service (admin variants in prose);
admin list_users + update_user_max_devices; progress history; book
matching; new password-rules doc.
Gate fix: authLabel missed JWT middleware written inline as
createJWTMiddleware(cfg) (uppercase 'JWT') — reported auth 'none' for
/api/auth self-service routes and would have for any wrapper-inlined
group. Labeled 'JWTMiddleware' + fallthrough 'jwt' case.
Claude registry: 12 entries. Drift fixed by gate: list_users on-device —
handler returns raw []database.Users (bare array, full model incl.
password_hash — real leak the API emits; pinned + documented; earlier
draft had invented {users,total,limit,offset} wrapper). max-devices
response is {message} only. books/match is GET /api/books/match
{matches, action}. progress history sessions = ReadingHistory rows
(pgtype fields). make test 9/9.
Scanner surface frontmatter'd (scan/start/stop/cancel/status/active/
active-summary + watch start/stop/status) with provenance comments;
active-summary on a non-admin group.
Claude registry: 10 entries. Fixed handler drift Beam missed: scan_library
request has no "recursive"; scanner/start binds folder_paths (not
invented interval_seconds); scanner/stop binds no body; response is
{message, job_id, status} with 409 coalescing variant
{error, active_job_id}.
Gate fix in drift_test seed: registerScannerRoutes group seeds used bare
"jwt"/"admin" tokens invisible to authLabel; and silently passed before
only because the seeded set was unchecked. Seeds now 'jwtMiddleware'/
'AdminMiddleware' markers. Also: cleanup of a stale-merge-script residue
in collections/update_rule.md example (extra null keys). make test 9/9.
Kobo emulation + KOReader calibre-compatible surface backfilled with
device-token auth; collections reorder + update_rule created.
Claude registry: 12 entries. kobo/koreader emulation endpoints all bind
arbitrary protocol POST bodies (markup/bookmark/gettests/sync-from-server,
koreader progress/bookmarks) — registered as AnyJSONObject, not bodyless as
first drafted. Initialization returns the library sync payload, not the init
response.
Contract correction in update_rule.md: the route is SetRuleEnabled ({enabled}
required -> {collection_id, is_smart}); Beam's draft invented rule-field
updates. Reordered response/status: {item_ids} -> {status: reordered}.
make test 9/9.
Beam backfilled 11 library docs (CRUD, folders, visibility, browse, stats,
media-items) and re-attached frontmatter to create_collection.md — which had
silently lost its pilot version in cc893f75: the negative-test
'git checkout --' restored the OLD file because the pilot was never staged,
and a frontmatter-less doc is invisible to the gate. Claude restored the full
corrected pilot content (priority higher-wins, time.String() note, 409).
Claude registry: 11 entries + one inverted catch — DELETE /:id/folders DOES
bind {folder_path} (handler reuses AddLibraryFolderRequest); registry first
draft said bodyless. GET /:id/visibility returns joined
GetUserVisibleLibrariesRow (type_name etc.), not bare Libraries.
Gate-driven merges fixed create/get library response shapes (sqlc row keys:
created_by_admin_id, library_type_id, updated_at; dropped invented type/
folders/is_visible). Fixed invented request fields (user_id in
set-visibility; library_type_id in update). make test 9/9.
Beam backfilled list/get/create/update/delete media-item, search, ratings
CRUD, progress GET/PUT. Beam correctly identified create/update/delete as
jwt+admin (admin group in the router — verified).
Claude registry: 12 entries. Notable pins:
- get_media_item: hand-built ~44-key map pinned verbatim from the handler
- create/update return the full database.MediaItems row (74 columns) —
examples merged up to the real shape via gate-driven merge script
- search returns []SearchMediaItemsRow (sqlc row — Real type pinned)
- rating scale is 1-10 int (odd = half-star); GET returns {rating: null} when
absent; update reuses the create handler (201)
- progress GET: 25 base keys + 4 CONDITIONAL position-authority keys
(css_selector/anchor_href/char_offset/context_text — added by the handler
for convertible formats with a valid CFI); PUT progress binds
sync.SaveProgressRequest and returns database.ReadingProgress
- Beam's examples were missing entirely on 4 files (list/search/progress
GET/PUT) — relabeled existing fences or inserted contract-shaped blocks
Gate-driven merge scripts (object + array variants) brought all examples to
the exact handler key sets. make test 9/9.
Beam backfilled auth core (login/register/refresh/logout/profile) and devices
core (list/update/delete/shelf CRUD/sidecar/register flow) with frontmatter,
provenance, and labeled examples. All claimed shapes verified against
handlers: AuthResponse {access_token, refresh_token omitempty, token_type,
expires_in, user}, update device {device_updated, device}, shelf wrappers,
DeviceAuthStatusResponse.
Claude registry verification: 15 entries pinned by type (LoginRequest,
RegisterRequest, AuthResponse, RefreshTokenRequest/Response, UserProfile,
DeviceListResponse, DeviceUpdateRequest, AddToShelfRequest, SidecarConfig,
DeviceRegistrationRequest/Response, DeviceAuthStatusRequest/Response).
Removed revoke_device.md — name-only duplicate of delete_device.md (same
DELETE /api/devices/:id route; was the last audit-orphan doc).
Gate catches this round (subtle Go serialization facts, now pinned in docs):
- uuid.UUID arrays never serialize as omitted: register/status response
ALWAYS carries device_id (all-zeros while pending) despite omitempty
- update device request binds device_name, not name
- register response includes device_id alongside registration_id
- list devices response is a {devices, total} wrapper
Beam added frontmatter + provenance + labeled examples to the 7 conflict and
6 queue docs, fixing real field drift on the way (resolve response keys,
bulk result shape, dismiss-all deleted count, queue stats counts).
Claude registry verification found TWO Beam catches of Claude's own draft:
- queue routes use :device_id / :item_id params (registry first draft said :id)
- resolve_conflict DOES take a body (ConflictResolutionRequest) — registry
first draft said nil
Both fixed; gate then surfaced one real Beam defect: example-request used a
nonexistent "resolution" field in resolve + bulk-resolve (real contract:
winner=<source|manual> with required manual_data for manual; strategy=
most_recent|highest_progress|manual). Fixed in the docs.
Pinned by type: ConflictListResponse, ConflictDetailResponse,
ConflictResolveResponse, BulkResolveRequest/Response, QueueStatsResponse,
ConflictResolutionRequest. dismiss-all deleted is an int count.
Beam drafted 15 frontmatter'd docs (saved-filters CRUD, dashboard
sections/preferences/restore, system config/settings, series list + books);
Claude verified shapes against handlers and pinned the registry:
- saved-filters: SavedFilterResponse pinned by type (RFC3339 timestamps here,
unlike most of the API); create/update share CreateSavedFilterRequest
(resource_type oneof media-items|collections|devices)
- dashboard: preferences GET pin {hidden_collections, collection_order,
items_per_section}; PUT returns full UserDashboardPreferences row; sections
wrapper {sections}
- system: GET+PUT /api/system/config are admin-defined key/value maps — new
AnyJSONObject sentinel (gate shape-checks without key comparison; request
side too). UpdateSystemConfiguration returns {status, message} — verified,
not guessed. GET /api/system/settings returns []SettingEntry.
- series: {series[{name, library_id, book_count, total_in_series,
cover_paths, last_entry_at}], total, limit, offset}; books by
?name=&library_id= with family-lens 403
Dedupe: deleted dashboard.md, system/config.md, system/settings.md — each
covered the same endpoints as Beam's per-endpoint files (content merged by
the gate-verified shapes; old prose recoverable from git).
Beam clarifications confirmed: scanner routes are WO5 (docs exist);
progress/books-match are WO5.
DeviceUsageResponse / PopularBooksResponse / services.JobResult pinned by
type; jobs create uses an inline anonymous request struct (RawMessage pin).
Two UNVERIFIED markers recorded (analytics date-range params, facet value
element shape).
New frontmatter'd docs:
- download (ServeFile — raw bytes, family-lens enforced; corrects the old
download doc's claim that no download route is registered: ServeFile IS
routed, DownloadBook is the dead code)
- set_audio_speed (speed range 0.25-4.0, NaN/Inf rejected)
- rescan (abridged example over the full 70+-column row; write-after-success
guarantee noted)
- purge-archived (destructive, admin), unarchive (204), merge (SHA-256
precondition, per-user collisions keep active copy)
- facets (two modes: facet+values, field autocomplete counts)
- filter_count (family lens)
Registry: RawMessage pins for map/abridged responses; Schema{} for non-JSON
body routes (download = raw bytes; unarchive = 204). Audit now:
missing docs down to ~55 (from 112), 0 orphans.
New frontmatter'd docs with real shapes (sqlc models pinned directly):
- bookmarks: get/create/update/delete (sync-aware rows: dedup_key,
last_modified_*, tombstones; delete is 204)
- review: upsert (empty string DELETES the review) / get (200 with
{review:null} when absent) / delete
- get single note + highlight (full sqlc row shapes)
- annotation history: restore (POST, kind via query or body) and purge
(DELETE, kind via required query; permanent)
Gate hardening forced by this batch:
- required-field walk now guards non-struct Request types (RawMessage pins)
- update_media_bookmark needed a full example block (no cross-reference
shortcuts — the gate checks every doc standalone)
New frontmatter'd docs with handler-compatible shapes:
- get_library_types.md (public; []LibraryTypes)
- list_libraries.md (admin; {data: ListLibrariesRow})
- get_library_folders.md (admin; []LibraryFolders)
- list/get stats/resolve/delete processing issues (admin; family-lens 403s
documented)
- scan_library.md (202 + 409 coalescing with active_job_id)
- get/update scan-settings (ScanSettingsResponse)
Gate improvements forced by this batch:
- Array responses: registered slice -> doc example must be a JSON array; key
set compared against the element type
- RawMessage pins may now be arrays
- update scan-settings uses a RawMessage pin: handler always sets Message but
the struct's omitempty would reject correct docs via zero-value marshal
Registry: sqlc models pinned directly (LibraryTypes, LibraryFolders,
ListLibrariesRow — json tags verified in the generated code).
Every API doc that declares frontmatter (method/path/auth/area) is now
checked by a unit test against three sources of truth:
1. Route registration: internal/router/*.go is parsed the way echo resolves
it (group prefixes concatenate; scanner.go's parameterized groups seeded).
Doc paths that no longer exist fail. (This is the check that would have
caught the /api/api/admin/hash-conflicts split.)
2. Auth: frontmatter auth label must match the middleware chain
(none|jwt|jwt+admin|device-token|optional-jwt).
3. Shapes: internal/docs/schema.go registry — example-request decoded with
unknown fields DISALLOWED (typo guard), required-by-validate fields must
appear (staleness guard), example-response top-level key set compared
both directions against the marshaled handler type (or a RawMessage pin
for hand-built map responses).
Five collections docs are the pilots with real handler shapes; notable
corrections they pin: update returns 201 (not 200), list excludes smart
collections unless include_auto=true, rule priority is 1-10 higher-wins
(docs claimed 1=highest), created_at is Go time.String() not RFC3339,
hide_finished/show_up_next toggles documented. CollectionResponse hoisted
out of GetCollections/GetBookCollections as CollectionSummary /
BookCollectionSummary so the registry can pin real types.
The gate fails on missing registry entries by design — that is the
maintenance contract. Known pre-existing (not this commit): vet copy-lock
warnings in media.go.
The admin group already sits under the /api prefix, so registering
"/api/admin/hash-conflicts" produced /api/api/admin/hash-conflicts while the
hash-conflicts page's htmx buttons posted to /api/admin/hash-conflicts/:id/
resolve — a 404 that htmx swallows silently (no swap, no error), so duplicate-
content resolution has been broken since the feature shipped (03cb4c78).
Verified live: sane path 401 anon -> 200 with admin token; doubled path 404.
Caller audit found no other consumers (Android app, koplugin, web reader,
tests, Bruno: no references). Docs already describe the sane path and are now
correct without edits.
Docs pages no longer embed the app Header (whose hx-post login form silently
degraded to a native GET, leaking credentials into the URL — this page never
loaded htmx). New docs-owned topbar: search, theme picker, Open App; anonymous
visitors get a plain /login link. Optional-auth middleware (hand-rolled —
echojwt's ErrorHandler return ends the chain, producing empty 200s) populates
the user for signed-in visitors.
Navigation rebuilt from the docs directory tree: 6 sections, subcategory
groups, server-rendered expansion and active states. Search results are a
templ partial (/docs/search/partial); code highlighting is server-side chroma
(previously dead hljs CSS); prose is theme-aware via typography DEFAULT vars
(prose-invert painted unreadable text under light themes). /docs renders the
index directly; standalone 404 without the tailwind CDN.
Tests: navigation routing/expansion (internal/docs), render pins asserting no
login form/password field/app sidebar ever return on docs pages.
lint-rendering: exempt *.test.ts fixtures (jsdom DOM building); docs.ts
baseline 5 -> 1.
Speed becomes a per-USER-per-BOOK preference that syncs across
devices, with a user-level default for books not yet played:
- new audio_playback_speed table (mirrors reading_speed's shape; a
preference, NOT a measurement and NOT a reading_progress column) +
users.default_audio_speed, schema in place
- chapters endpoint resolves per-book row → user default → 1.0 and
ships playback_speed + default_playback_speed (zero extra round
trips; default powers the dropdown marker)
- PUT /api/media-items/:id/audio-speed upserts (range-checked; the
client SPEEDS list stays the real gate; library visibility gates
writes)
- the user default rides the profile path like theme/timezone:
profile form gains a 'Default Playback Speed' select (server-side
SpeedOptions mirrors the web SPEEDS list), GET /profile carries it
- web: restorePosition applies the resolved server speed (localStorage
audio_speed demotes to a first-paint cache); setSpeed PUTs
fire-and-forget with a failure toast; the Speed dropdown gains a
'(default)' row marker and a 'Make N× my default' action with a
success toast
ALSO fixes a pre-existing profile-page bug surfaced by the §8 form
verification: the password section's current_password input was
required INSIDE the shared form, so HTML5 validation blocked every
Save Changes submission (dead since the shell migration). The
password flow keeps its own gates (disabled-button checkers + server
400).
Integration test TestAudioPlaybackSpeed pins the full contract (unset
→ 1x, upsert resolution, range rejection, default fallback for
row-less books, default updates re-satisfying unset books). Verified
live: 9 Playwright checks.
PUT /api/media-items/:id (the metadata editor) writes the FULL row via
UpdateMediaItem, whose query sets audio_metadata unconditionally — and
audio_metadata has no editor field, so the params literal left it nil
and every save NULLed the payload. A plain metadata edit (genre tweak,
title rename) dropped any audiobook back to "1 second long" with no
track manifest: the same incident class as the scanner wipe, sitting
on the most-used admin path in the app.
Carry existing.AudioMetadata through the write. Write-after-success:
only the audio chain (probe-backed extraction /
UpdateMediaItemAudioBookPayload) ever replaces that column.
The 2026-10-21 wipe: a re-extraction pass NULLed audio_metadata on the
m4b audiobooks — every affected book dropped its chapters, track
manifest and duration and displayed as 1 second long (the reader also
lost its manifest, since single-file books synthesize it from the
payload).
Guards on both writers of that column:
- updateMediaItemWithMetadata (generic full-row pass): when the new
extraction carries no audio payload, keep existing.AudioMetadata.
Single-file m4b books route here, and a failed/absent extraction
leaves AudioMeta nil.
- updateAudioBookItem (focused payload write): an empty payload (no
tracks, no chapters, no duration) over a populated row is a failed
extraction — refuse the write with a loud warning and let the next
scan retry.
audioPayloadIsEmpty() pins the empty-payload predicate (unit-tested):
the column is only ever replaced by fresh data from a real audio
extraction, never by absence of one.
Force rescan contract change (owner-directed): the editorial-keep skip
is retired. Force rescan now re-derives EVERY field on EVERY book from
all sources — multi-file books included (the folder/zip flows ran the
full chain on create and on manifest-flagged edits, but skipped the
sidecar apply under force). metadata_overrides is the only protection:
ApplyMetadataOverrides re-applies manual values inside the writer, so
manual edits survive; everything else re-derives.
- A force pass also STAMPS each re-derived book's manifest with the
current parser version (success-only — same truthfulness rule as
Phase 10) and closes its open processing issues (Phase 11 self-heal).
Force rescan and the ev-bump trickle now converge: either leaves the
library fully current; the deploy order is bump-first, then Rescan
All (documented in the plan and contract_pins).
- Single-file force rescan already ran the full chain — unchanged,
now pinned.
NEW filter field manually_edited ("Manually edited", boolean):
cardinality(mi.metadata_overrides) > 0 / = 0. Composed-engine (qf)
only — the builder's Book facts group gains it with is true/false;
no value menu, no suggestion surface; family lens inherited.
Tests: multifile force applies sidecar metadata + stamps + override
protection; single-file pin; the filter both directions scoped to the
test library; adjacent suites (Audiobook, Calibre, ExtractorVersion,
CrossLibraryMove, ProcessingIssues, FilterBuilder, SchemaIdempotency)
green. The extractor mechanism itself is untouched — the owner bumps
scan_extractor_version manually via the admin UI (plan Phase 13 §3).
processing_issues only recorded cross-library move rejections; files
that failed to import vanished into an error counter and stdout. The
ledger now records every failure class and heals itself:
Schema (in place, pre-release boundary):
- media_item_id NULLABLE + file_path TEXT (deliberately unbounded —
an over-long path is itself a failure mode; a VARCHAR(500) ledger
column died on the exact 22001 it existed to report — caught by the
new test). Row-less failures key on (library, file_path); item-backed
on (item, type). Widened dedup = COALESCE-expression unique index;
the old table UNIQUE stays (NULLs pairwise-distinct for row-less).
- Conflict-target upsert: repeat failures UPDATE the same row and
REOPEN resolved ones — never duplicates.
Wirings:
- CreateMediaItem failure (single-file + multifile createAudioBookItem)
→ row-less import_failed (error).
- Extraction hard-fail with fallback import → item-backed
metadata_extraction_failed (warning).
- Change-path re-extraction failure (the ev/sidecar guard) → item-backed
metadata_extraction_failed; LogProcessingIssue keeps its signature, a
row-less helper joins it.
Auto-resolution:
- Successful import → ResolveOpenProcessingIssuesForPath (library,
file_path).
- Successful (re)extraction → ResolveOpenProcessingIssuesForItem
(single-file change path, multifile editorial apply).
- Unchanged-skip scans do NOT resolve: extraction never re-ran, so the
degraded metadata is still degraded — the page heals exactly when the
book does.
Family lens on ALL issues surfaces (list, stats, resolve, delete, SSR
page): explicit out-of-set library id → 403, never silent. This
tightens one pre-lens behavior — a nonexistent library id returned
200-empty, now 403 (pinned, changelog entry in docs/contract_pins.md).
List is LEFT-JOINed so row-less rows appear (media_item_id "",
title falls back to the path).
Tests: full lifecycle state machine through real scans (fail → appears,
repeat → no dups, success-at-path → resolved, LEFT-JOIN list shape);
family lens 403s with a fresh non-shared admin; all 7 pre-existing
issue suites green (two expectations widened to the lens, changelogged);
adjacent suites (CrossLibraryMove, Audiobook, Calibre,
ExtractorVersion, SchemaIdempotency) green; unit suites green.
Parser fixes (7e4c97f6, genre model v2) never reached rows ingested
under an older parser: the sidecar never changed, so every fingerprint
said 'unchanged'. The manifest JSONB now carries the reserved 'ev'
stamp — the parser version that produced the item's row.
Semantics (all pinned by tests):
- ABSENT ev decodes as 1 ('v1-or-unknown'); stamping is truthful-only:
create paths + post-extraction stores stamp the setting's current
value; deploy-day backfill and the extract-failure guard stay UNSET
(a row never claims a version its data can't prove — a mid-backfill
bump self-heals next poll); repoint reconciliation preserves the
stored ev (stat-only refresh re-extracted nothing).
- NEW setting scan_extractor_version (seeded + registry + getter,
int default 1, category scanner): a manifest whose ev != the setting
re-extracts through the SAME change path (full extraction →
updateMediaItemWithMetadata → fresh stamped manifest). Sidecar-less
books get an entries-less manifest as their ev home (create: stamped;
first-sighting: unset).
- Catch-up bounded: evReextractBudgetPerScan=100, atomic (watcher pass
can overlap a poll scan), reset per ScanFolders — amortized +
resumable across polls; no force rescan, no flood.
- Multifile books included: the ev gate rides multifileSidecarChanged's
shared ladder; ev-driven manifestChanged drives the editorial apply
+ stamped re-store.
Settings API: the unified GET/PUT /api/system/settings now lists the
key (registry-driven, auto-surfaces in the admin UI). Test harness:
the registry is now Loaded + wired into the SystemSettingsHandler
(main.go parity) — settings writes through the API were 503s before.
Verified: unit (ev normalization, budget CAS, GetExtractorVersion
cache paths, registry clamping); integration state machine —
create-stamp ev=1 → equality skips (corruption probe persists) →
wiped scan-state backfills UNSET without re-extract → bump re-extracts
ONCE (sidecar value restored, ev=2) → stable (second corruption
persists); multifile twin green; adjacent suites (Audiobook, Calibre,
CrossLibraryMove, SchemaIdempotency) green; live app restart re-scans
1198 files with 0 changed (deploy-day rule), setting listed at
/api/system/settings.
The browse-on-click blank sentinel was trimmed to empty (correct), but
the typed query had already been bound unconditionally — the id-source
suggestion query has no score CASE to reference it, leaving an unused
parameter Postgres rejects (42P18). Every click of an empty library /
collection / added-by box answered 'No matching values' via the error
path.
- ComposedFieldValues binds the typed query ONLY in branches that
reference it and only when non-blank; the score expression is
'0 AS score' for browse (no param) and the similarity CASE when
typing. composedIDSourceValues takes the raw typed string and binds
it only when the name filter is built. userBind already followed
this discipline; audited every bind site in the file (fuzzy helpers,
composed search, count) — all interpolate immediately.
- Tests: library browse under rules (exactly the surviving library,
count pinned), series browse blank, added_by browse blank (no
error), collection browse blank (offered with count) and under a
zero-match rule (never offered). Verified end-to-end against the
live dev data: author+Sapkowski AND narrator+Kenny → Audiobooks (8).
- Blank typed queries rode the STATIC autocomplete path for the classic
six fields, where a single space similarity-matches into nothing —
clicking an empty value box with no other filters said 'No matching
values' in a library full of values. Blank = browse now always rides
the composed engine (which trims and answers the top values); static
keeps its pinned contract for real typed queries. Regression-pinned:
browse with no rules lists the field's values.
- The value popup now drops BELOW the input (top: 100%), not over it.
- Field-menu group headers lost position:sticky — the search box alone
stays pinned.
- Per-rule connectives: the chip before each rule flips that rule
between AND and OR — OR-joined neighbors merge into an invisible
OR-block, AND pulls them back out. The tree stays the single source
of truth (implicit blocks are a client marker the server ignores),
so deep links, saved filters, and the composed engine are untouched.
Group-mode pills are gone; explicit groups remain as pure structure
(+ group). Compile limit, same as Excel: flat chips express
AND-of-OR-groups; a top-level OR of AND-blocks still needs nesting.
- Browse-on-click: clicking an empty value box now fetches the field's
top values UNDER the other rules (blank typed = browse, not a
similarity filter) — 'assign an author, click Publisher, see their
publishers' without typing. Server: the composed suggestion engine
trims the typed query; client: the >=2-char guard no longer blocks
the empty query, browse limit 15 vs 8 while typing.
- Tests: the browse case pinned (blank sentinel + rules → the field's
values under those rules, zero-match values absent).
- Field registry (services/filter_registry.go): ~55 typed fields —
every book column, file/lifecycle facts, the three HOLDING
STRUCTURES (library, collections with excluded-aware membership,
series with FULL primary+alternate_info membership), and the whole
user-relative class (status, progress, last read, my rating, review,
notes/highlights presence, note/highlight text, time spent, pages
read, collections, added by) reading the VIEWER's rows. Operators
are typed constants; negation is an operator.
- Composed engines (services/filter_query.go): one Go clause builder
over raw pgx feeds the search (identical SearchMediaItemsUnifiedRow
shape), the field-aware suggestions, and a true-COUNT live count.
Mandatory WHERE head everywhere (family lens + integrity facts).
Values are always bound parameters — no user string reaches a
field/operator position, ever.
- Wire: the qf JSON param (validated strictly — unknown
fields/operators/malformed values are 400); qf wins over flat params
on hand-crafted URLs; flat deep links keep working unchanged.
- Endpoints: GET /api/media-items/filter-count (live count);
field-values autocomplete upgraded in place — any registry field
key, qf binds the OTHER rules (symmetric under-filter suggestions,
zero-match values never offered); classic six without a tree keep
their static pinned contract.
- UI: the drawer becomes the rule builder — the ONLY filter UI (templ
markup, Alpine tree state, sanctioned island; grouped field menus,
operator menus with red negations, field-aware value menus, live
count, chips per rule with red negations, URL rehydration, smart
serialization flat-when-simple else qf).
- Tests: composed semantics (AND/OR/negation/nesting/enums/bools/
numbers/dates/empty), every user-relative field class with the
my-data weld (two readers, different worlds), note-text substring
pin (catharsis ≠ Carthage), family lens (hidden libraries nowhere,
403s), suggestion-under-rules exactness, qf validation + precedence,
flat-vs-tree equivalence; registry completeness unit test.
- Remove the Phase 7 facet-panel UI: SSR route, facet_panel.templ,
FacetCountsForView, drawer panel block, panel TS + .facet-* CSS.
The ENGINE stays (FacetCountsUnderFilters, all-facets endpoint,
drift guard) — it becomes the Phase 9 builder's machinery.
- Fix the All-Books bug: field-values autocomplete (author/genre/
tags/narrator/series/language) treats library_id as OPTIONAL —
absent aggregates every VISIBLE library (family lens in-query);
explicit out-of-set library_id is now 403, never a silent pass.
The four scalar Search*Values queries get the nullable shape.
- toast.ts: suggestion fetches are exempt from error toasts
(console-only) — a failed background hint never interrupts.
- TestFacetPanelPartial deleted with the behavior; TestFieldValuesAll-
Libraries pins the fix (200 + cross-library aggregation, family
lens, 403). Contract-pins changelog entry added.
The 9 Batch A facets could not be SET from the web UI, the facets
endpoint had no in-app consumer, and the registry's libraryTypes hints
rendered nowhere. This phase closes all three.
- ONE UNION-ALL sqlc query (FacetCountsUnderFilters) computes all 14
listable facets in a single round trip: each arm omits its OWN
facet's filter block and carries every other registered facet's
block (copied mechanically from SearchMediaItemsUnified) — classic
faceted semantics: with filters active each facet counts under the
others; with none, blocks are neutral and the counts are the v1
global-per-library ones. One query serves both cases. The 14
per-column Batch C queries are deleted.
- Endpoint: no facet param → the new all-facets shape
({facets: {param: [{value, display?, count}]}}), filters bound via
the shared ParseMediaSearchParams, library_id optional (the
All-Libraries view, family lens still enforced); facet=X keeps the
pinned single-facet shape (now routed through the same query with
neutral filters).
- SSR partial GET /media-items/facet-panel: templ renders the drawer's
value lists — registry order, Batch D library-type scoping
(FacetsForLibraryType: manga_type on manga, reading_direction on
manga+comics, narrator on audiobooks; All view = union), prettified
genre labels + language display names, active values marked. Family
lens: out-of-set explicit library_id is 403.
- Drawer wiring: the panel fetches on drawer open and after every
apply with the CURRENT filter query string; a value click toggles
that facet's comma-joined filter (hidden input where the drawer has
no field — same as deep links) and re-applies the form; chips and
URL stay in sync. Values are server-rendered (rendering doctrine);
bookshelf.ts owns only the click/fetch orchestration.
- prettifyGenre moves to internal/utils as the canonical
implementation (services' display enrichment shares it; templ wraps
it; the TS port in bookshelf.ts is unchanged).
- Drift guard extended: FacetCountsUnderFiltersParams must carry every
registry facet's param, the bools as pgtype.Bool.
Tests: under-filter exactness (own-block omission, OR-within,
AND-across), family lens (hidden values absent; 403 on both shapes),
all-shape + pinned-shape coexistence, partial rendering with Batch D
scoping + active marking + 403. Full container suite green except the
documented pre-existing TestAdminCreateUser red.
Reverses the Phase 4 scope cut: for EXISTING folder/zip audiobooks a
sidecar edit was invisible to everything, including force rescan —
updateAudioBookItem writes only scan-derived fields, and these formats
often have no useful embedded tags, so the sidecar is the only source.
- Change fingerprint gains the manifest compare in both processors:
changed := size differs || track payload differs || manifest drifted.
The ladder runs against the bulk-loaded map entry (stat pairs only,
zero extra queries); NULL stored manifest backfills without
re-extraction (deploy-day rule, same as single-file).
- Manifest-changed applies the once-extracted book metadata to the
editorial columns (updateMediaItemWithMetadata, overrides protected)
and re-runs the playlist hook. The apply runs BEFORE the focused
payload write: the generic update's format-group pass derives mime
from the sentinel (none) and chapters from metadata.ChapterCount
(unset) — the payload write must win (rescanMultiFileAudioBook's
ordering). Track-only changes keep the existing editorial-keep
update; force rescan keeps it too.
- Manifests seeded at multifile create (folder names derive from the
sentinel — the exact representative the extractor reads — via
detectSidecarChangeWithRepresentative, so manifest and extractor can
never disagree; a directory-basename cover is ignored by the
extractor and therefore not tracked).
- Zip parity fixes: the fast path now lifts archive state like the
folder path always did (a returned zip book stayed archived
forever), and the change branch handles the unchanged-archived case.
- Single-file sidecar change path: extract-failure guard — the old
non-nil empty &MediaMetadata{} fallback bypassed the update path's
filename-title guard (preExtracted == nil only) and would blank the
title on a failed extraction; the row is now left untouched and the
manifest refreshed. Empty-success extraction (sidecar removed from a
tag-less book) now passes nil so the create-equivalent chain (tags →
filename) re-derives instead of blanking.
- Multifile changes now count in the scan summary (changedItems);
they were previously uncounted.
Tests: sentinel-name derivation unit pin; folder sidecar edit
(title + collection sync + fresh manifest + steady-state skip),
track-only change keeps editorial, zip sidecar edit + archive
restore, sidecar removal falls back to filename (the wipe-guard's
testable sibling). Full container suite: green except the documented
pre-existing TestAdminCreateUser red.
Part 2 of the faceted plan, Phase 5 (steps 5-6): the scan that used to
print ~4 lines per unchanged file now prints ONE summary line, and a
poll tick landing mid-scan no longer stacks a second full scan.
- Quiet scans (§4): the walk loop is silent — inserts, changes, and
errors log inside processMediaFile; the duplicate
'Successfully processed' branch (both branches printed it) and the
per-file 'processing / already exists / same size, skipping' chatter
are gone. Summary carries the trigger and a changed counter:
'Scan of <folders> (trigger: poll): N files, N changed, N new, Xs,
N errors'. Triggers threaded through every enqueue site: poll
(startBackupScan), watcher (dirty-dirs flush), admin (scan API),
initial (startup scan).
- Enqueue coalescing (§5): Worker.HasActiveScanForLibrary (JobResult
gains additive job_type so only SCAN jobs coalesce — an import
sharing the library id never blocks one). enqueueLibraryScan skips
silently; the admin scan API answers 409 with the active job id.
Enforces the one-scan-per-library invariant the bulk scan-state map
leans on.
- Watcher capability probe (§4): ONE Add on a root folder; failure
disables the watcher with a single startup line (replaces the
per-directory warning spam on inotify-broken deployments). The
watcher stays an optional latency reducer over scheduled scans.
- Per-directory sibling cache (§6): hasSiblingBookFileCached caches
the ReadDir per directory for one scan — loose-image manga re-stats
thousands of images sharing a handful of directories.
Tests: TestHasActiveScanForLibrary (unit — running/pending coalesce,
terminal jobs don't, non-scan jobs don't, empty library never, nil
worker safe). Full scanner suite green in the container.
Part 2 of the faceted plan, Phase 4 (steps 3-4). Size-only detection
was blind to metadata-sidecar edits; a size-stable book is changed by
EDITING its metadata.opf/metadata.json/cover — previously invisible
until a force rescan.
- Sidecar manifest fingerprints (media_scanner_manifest.go): the
manifest stats exactly the names the extractors consume (metadata.opf,
metadata.json, findSidecarCover patterns incl. <basename>.<ext>), so
manifest and extractors can never disagree. Shape v1:
{v, entries:[{n, s, m (unix nanos — second granularity would hide
edits landing in the backfill scan's own second), h (sha256 — covers
included: the repoint ladder needs a baseline hash for any stat-
drifted sidecar; a hashless cover turns harmless mtime drift into a
re-extraction, the exact flood this design kills)]}.
- Evaluation ladder in cost order: stat pairs identical → skip with
ZERO bytes read; stat drift → hash that sidecar (equal = host repoint
reconciliation, stat pair refreshed, no re-extraction; differing =
changed); NULL stored manifest → backfill only, NO re-extraction
(deploy-day rule: the first scan after release must not look like an
mtime flood).
- Change path: re-read full row, extractMetadata once and pass it
through updateMediaItemWithMetadata (preExtracted — metadata_overrides
stay protected), run syncPlaylistCollectionsAfterCreate (Correction 2:
a reading-order entry added to an EXISTING book now creates its
collection; add-if-missing is idempotent), store a fresh full
manifest. Creates seed their manifest at insert time.
- Multifile folder/zip books intentionally keep Phase 3 behavior: their
change path does not consume sidecar extractors, so gating them would
flip books 'changed' without picking anything up (plan correction —
the plan's §3 multifile note assumed otherwise).
- UpsertMediaScanState + GetMediaScanState queries.
Tests: media_scanner_manifest_test.go (ladder unit tests — fast path,
repoint, same-size drift, appear/vanish, unhashable legacy entries);
TestScanManifestLifecycle (seeding, unchanged-skip stability, repoint
reconciliation with stat-pair refresh assertion, same-size edit
consumed, playlist sync on the change path + idempotency, no-leak
guard). Container runner.
Part 2 of the faceted plan, Phase 3 (steps 1-2). The periodic scan
issued one GetMediaItemByFilePath per scannable file (~3,094 queries
per scan in prod, nearly all answering 'unchanged, skip') plus one
GetLibraryByFolder per file for folder resolution.
- NEW media_scan_state table: the scanner's per-item fingerprint
cache. Deliberately NOT a media_items column — the library-scoped
list emits raw sqlc rows as JSON (7537063c), so scanner bookkeeping
must never leak into API payloads nor churn the model or pins.
Manifest JSONB shape v1 documented in the schema (n/s/m/h; ev
reserved for the Phase 4+ extractor-version mechanism).
- GetScanStateByLibrary: lean bulk projection (id, file_path,
file_size, archived_at, missing_scan_count, file_sha256, manifest).
archived/hash columns ride along so the skip path's archive-restore
and hash self-heal checks stay DB-free too (refines the plan's
4-column sketch; same goal, narrower invariant).
- ScanFolders: per-run reset, per-folder library cache, one bulk load
per distinct library. processMediaFile: map hit answers existence +
size + flags with zero DB reads; force-rescan and size-change
branches re-read the full row (the change path may be expensive);
write-through on create/size-change keeps later passes honest.
- Multifile folder/zip books keep their full-row lookup: it is one
query per BOOK ROOT (not per track) and needs audio_metadata for
the duration-probe check. They join the map in Phase 4, when the
manifest compare replaces their fingerprint check.
- nil map (watcher's scanDirectory path, bulk-load failure) falls
back to per-file lookups — behavior-preserving; the failure case
logged once per scan.
Test: TestScanStateBulkLoad (unchanged rescan leaves updated_at
stable; API-created rows are seen by the bulk load; size changes
still update in place). Container runner.
Phase 1 + backend of Phase 2 of the faceted-navigation plan. One
declarative facet registry (services/facet_registry.go) drives the
bookshelf filter system; a drift-guard unit test reflects over the
generated SearchMediaItemsUnifiedParams so a registry entry and its
sqlc.narg block can never disagree silently.
- Nine new filter params on SearchMediaItemsUnified: story_arc,
imprint, edition (scalar-fuzzy), age_rating, format_group,
manga_type, reading_direction (exact), explicit, abridged
(presence-bound bools). Comma = OR-within multi-select on every
text facet; AND-across-facets unchanged.
- Genre model v2 (user rule: a subgenre is not a genre, it's a tag):
genre_lexicon.go rebuilt as shelf set + subgenre->shelf map;
pickSmartGenre picks direct shelf > mapped subgenre > Fiction floor
> empty (unknown-only stays tags-only). Type-words (Manga, Comics,
Graphic Novel) join the demote list. Both 'adventure' and 'action &
adventure' are shelves — provider data uses both and the verified
traces pin them.
- Language filter UX: comma elements normalize through the ingest
language map, so display names ('English') match rows stored as ISO
codes ('en'). The language autocomplete now serves display names.
- NEW GET /api/media-items/facets?facet=X&library_id=... — registry-
driven value+count listing (14 static per-column queries, family-lens
join); language values carry display names; bool/range facets 400.
- Family-lens gates: an explicit hidden-library library_id is now a
403 on the search endpoints (was a silent empty set) — JSON branch
and HTMX grid partial both.
- handlers.ParseMediaSearchParams is the SINGLE param parser for the
search JSON endpoint and the HTMX grid partial; the HTML branch
previously dropped publisher_filter and narrator_filter silently.
Pins: docs/contract_pins.md changelog entries (additive request
params; new facets endpoint). Tests: facet_registry_test.go (shape +
SQL drift guard + library-type union), series_parsing_test.go genre-v2
cases, facets_integration_test.go (param filtering e2e, counts,
language display, 403s) — container runner.
Completes the A2c acceptance scenario: click an author on a book
detail page -> land on the bookshelf filtered to everything that
author wrote (including their share of co-authored books).
- Media detail + unscoped list resolved maps expose subtitle,
explicit, authors, publishers (the library-scoped raw branch and
the search endpoint already did — raw rows follow the SELECT).
- SearchMediaItemsUnified SELECT carries the new columns; the search
endpoint serializes the row directly, so clients get them there too.
- book_detail.templ renders each author as a link to
/bookshelf?author_filter=<name>.
- Bookshelf honors filter params in the URL on load (author_filter,
series_filter, genre_filter, publisher_filter, tags_filter,
narrator_filter, language_filter): pre-fills the form, applies it to
the first grid load. Comma-joined values are multi-select via the
server-side string_to_array split. Filtered views are shareable.
Pins widened: search rows (+7 keys), resolved keys (+4); changelog
entries in docs/contract_pins.md. Android follow-up plan written to
~/.opencode/plan/bookhoard-app-v2-compat.md.
Phases 4 + 5 of the metadata overhaul (plan §4, §5) plus §3 handler
wiring.
Filters (§4.1): author_filter/publisher_filter now match per-element
against the authors[]/publishers[] arrays — a co-authored book matches
any one of its authors — with comma-joined params acting as
OR-within-facet multi-select (string_to_array + cross unnest). Legacy
single-value params unchanged on the wire; series/genre/language
filters stay on their varchar columns; narrator filter already
per-element.
Collections API (§3.3-3.5): collection detail carries viewer +
hide_finished into the listing query and exposes up_next (visible-
library-enforced reading-queue head); PUT
/api/collections/:id/items/order (owner-only, full-order rewrite);
UpdateCollectionRequest gains hide_finished/show_up_next ([]
string last-value-wins for the hidden-false-sibling checkbox pattern)
merged server-side OVER stored view_settings so imported_from_series
provenance survives edits; EditCollectionModal renders the toggles
from view_settings.
Dashboard (§5.5): sections JSON carries is_playlist/complete
(dashboard.ts is the AGENTS-documented JSON-render exception; first
paint stays templ). Fully-read playlists render '✓ Playlist complete'
instead of an empty rail; imported playlists get a 📖 Playlist badge.
Frontend: book detail shows subtitle + full authors + Explicit badge;
edit modal gains subtitle, comma-separated authors/publishers (client-
split to arrays exactly like narrators), step=any series number,
explicit checkbox; collection book cards get playlist move ↑/↓ buttons
persisting the full order via the reorder endpoint (templ partial
re-render on failure — DOM order is client-local, API order is truth).
AGENTS.md gotcha #3 amended per plan §3.7: user collections are
dashboard sections by default; the show_on_dashboard premise is
retired.
Verified: full affected integration sweep green (pins widened for
up_next/is_playlist/complete — changelog entries in
docs/contract_pins.md); unit suite green; go build + templ + vite
green.
Phase 2b + 3 of the metadata overhaul (plan §2.4, §3.1-3.7).
Auto-playlist collections (§2.4): the scanner turns obviously-playlist
sidecar series (explicit '(Reading Order)'/chronological/publication/
in-order/timeline markers only — precision over recall) into curated
static collections: upsert by view_settings.imported_from_series then
name (rename-safe), position set ONLY on insert (clean integers; 1.5/1a
stay unpositioned rather than guessing), membership add-if-missing,
never removed or re-ordered by rescans, provenance in description +
view_settings, show_on_dashboard true (dashboard-eligible by design),
'scanner.auto_playlist_collections' setting (default on, Settings
Registry + schema seed).
Ordering (§3.1-3.2): collection listings order position ASC NULLS LAST
then added_at ASC (was added_at DESC — walked playlists backwards);
PUT /api/collections/:id/items/order applies full-order rewrites
(owner-only, service-gated).
hide_finished + Up Next (§3.3-3.4): GetCollectionItemsForDashboard
gains viewer_user_id + hide_finished (LEFT JOIN reading_progress,
COALESCE percentage < 0.999) and viewer_percentage rides the row for
card graying; GetCollectionUpNext serves the collection's reading-queue
head with the family-lens visible-library array enforced.
Dashboard (§3.5): user collections are dashboard-ELIGIBLE BY DEFAULT —
the show_on_dashboard gate (never writable for user rows since the
column shipped) is gone from GetUserCollectionsForDashboard; hiding
stays dashboard-prefs (hidden_collections) territory; hide_finished
filtered-everything now emits a Complete section instead of silently
vanishing; sections JSON carries is_playlist/complete.
Series page ordering hardening (§3.6): GetSeriesBooks tiebreaks by
title ASC — NULL/tied numbers previously returned heap (scan) order,
which is what the user saw.
Frontend custom-section-builder note: the builder has sent
show_on_dashboard: true since its creation; the API never bound it.
User-row visibility is now intentionally prefs-governed, so the field
stays unbound by design (documented in code).
Pins widened (docs/contract_pins.md): collection detail + up_next;
dashboard sections + is_playlist/complete; new reorder endpoint
recorded.
Verified: TestPlaylistCollectionsEndToEnd (new, full e2e: sidecar
series split, collection auto-creation with position 7, rescan
idempotency, owner-only reorder, finished-filter + completion marker +
Up Next drain) plus the full affected integration sweep green.
Phase 2a of the metadata overhaul (plan §2.1-2.3, §2.5-2.9).
The headline bug: ABS writes sidecar series as an array of STRINGS
(["The Witcher #3", ...]); the parser only accepted {series, sequence}
OBJECT arrays, so every ABS-matched book silently lost its series (found
live: 9 sidecars, zero series ingested; only folder-structure Malazan
survived). parseSeriesCandidates now normalizes string arrays, object
arrays and Libation plain strings.
Three-way series assignment (plan §2.2):
- primary bibliographic series -> series/series_number columns
- additional real series (Drizzt: Icewind Dale #1 = Legend of Drizzt #4)
-> alternate_info JSONB, comic-established shape
- ordering constructs ('The Witcher (Reading Order)') -> PlaylistSeries
candidates; NEVER the series column. Detection is explicit-marker-only
((reading order|chronological|publication order|in order|timeline) as
parenthetical/suffix) — precision over recall; a series merely
containing 'timeline' mid-name stays a series.
Sequence policy: leading-number extraction ('Book 2' -> 2, '2,
Dramatized Adaptation' -> 2, '1a' -> 1); lossy extractions preserve the
raw text in alternate_info.series_sequence_raw.
Smart genre (deterministic lexicon): compound Audible blobs split
('Epic, Action & Adventure, Military, Sword & Sorcery' -> 4 candidates),
alias-normalized, format noise demoted ('Audiobook' never wins),
most-specific canonical genre takes the genre column, all candidates
stay in tags.
Language: display names normalize to ISO 639-1 on ingest; ROOT-CAUSED
the update-path drop — CreateMediaItem call site never set Language
(explicit NULL over the column default), the update path did; create
now carries it.
Title-derived series fallback (§2.7): 'Baptism of Fire: The Witcher,
Book 3' 2-level folders split conservatively (only when the post-colon
tail matches ', Book N'), subtitle parsed for series+number, blanks-only
precedence, matching-name number fill, zero display mutation. Works
with no sidecar and no network.
Ingest fixes: subtitle/explicit land in their columns (audio tag
subtitle stops joining into title); full authors[] array (was [0]-
only); sidecar descriptions sanitized via bluemonday UGC policy
(vendored).
metadata_overrides: verified scan-update path already applies
ApplyMetadataOverrides; subtitle/explicit/authors covered by the Phase 1
override adaptation.
Tests: new series_parsing_test.go (string-array sidecar e2e, Drizzt
alternate_info, lossy raw preservation, playlist classification,
title-derived fallback incl. no-mutation + blanks-only, genre lexicon,
language map); audio-tag subtitle pin updated to column behavior;
calibre/ABS 2.5-series_index pins widened per float change (changelog'd
in phase 1).
Deferred to 2b: auto-created playlist collections (§2.4) — settings
toggle, collection upsert, ownership.
Phase 1 of the series/collections/metadata overhaul
(~/.opencode/plan/series-collections-metadata-overhaul.md).
Schema (in place, idempotent):
- media_items.series_number INTEGER -> DOUBLE PRECISION: real-world
ordinals are decimals (0.5, 1.5 novellas); INTEGER silently truncated
them (calibre 2.5 pinned as 2 in tests).
- + subtitle TEXT, explicit BOOLEAN (sidecar ingest; ABS explicit /
Audible isAdult).
- + authors/authors_search/publishers/publishers_search TEXT[] NOT NULL
(narrators A2a precedent). Arrays are write truth; author/publisher
varchars become trigger-regenerated display strings
(sync_author_publisher_display) so every varchar consumer (trgm
search, facets, API, OPDS/Kobo) keeps working unchanged.
- Row-level NULL normalization for the new arrays in
set_library_type_name (A2 lesson: Go nil -> explicit NULL -> 23502).
- Backfills ARRAY[author]/ARRAY[publisher] — deliberately no
comma-splitting ('Tolkien, J.R.R.' would corrupt; scanner kept only
authors[0] historically).
- collection_items.position INTEGER + (collection_id, position) index
for playlist ordering.
Go: sqlc regen; MediaMetadata/handler params gain Subtitle/Explicit/
Authors/Publishers (legacy singles still accepted, arrays win);
SeriesNumber int32 -> float64 end to end; metadata_overrides adapted
with legacy override keys ('author'/'publisher') preserved so existing
user overrides survive; bulk author edit writes the array.
Pins widened (docs/contract_pins.md): raw sqlc row key set + the six
new columns; changelog entry added.
Verified: unit suite green; integration sweep (bulk/smart collections/
scanner/stats/sync/queue/calibre/pins/audiobooks) green against the
shared dev DB after in-place migration. Pre-existing unrelated red:
TestOwnershipAndAdminTestSuite/TestAdminCreateUser timezone inheritance
fails identically at the pre-change commit (live shared state).
John O'Keefe