bookhoard/docs/contract_pins.md

Wire-contract pins (hardening batches)

These are the response SHAPES (JSON key sets, not values) frozen by the TestContractPin_* suite in cmd/server/tests/contract_pins_test.go (batch H0). The plan they serve is ~/.opencode/plan/hardening-canonical-types.md.

Rules

1. Frozen: no existing response key may be RENAMED or REMOVED on any endpoint below (JSON or KOReader/Kobo adapters). Requests are frozen too — no parsing changes.
2. Additive keys are allowed and expected — but a batch must widen the allowed slice in the test AND record the addition in the changelog below, in the same commit.
3. KOReader / Kobo / progress surfaces are the hard freeze: the KOReader plugin is an external client whose handlers keep response shapes byte-identical (only their internals may route through shared funnels).
4. A behavior fix that would require narrowing or REMOVING a pinned key needs the user's explicit sign-off before the pin is touched — a pin freezing buggy behavior is a known-bugs ledger, not a license.

Pinned surfaces

Test	Surface	Shape
ProgressWeb	GET /api/media-items/:id/progress (percentage-only write)	flat map: id, media_item_id, user_id, current_page, total_pages, last_read_at, percentage, character_offset, epubcfi, chapter, chapter_progress, format_group, total_characters, chapter_count, last_sync_device, last_sync_source, last_sync_timestamp
	… conditional bolt-ons	context_text (when stored valid); css_selector, anchor_href, char_offset, rewritten epubcfi (when the anchor verify path fires)
	PUT /api/media-items/:id/progress response	database.ReadingProgress JSON (25 keys incl. audio_offset_seconds, viewport/zoom/scroll/panel/reading_mode)
	GET with no rows	exactly {current_page, total_pages}
KOReaderProgressPush	POST /api/sync/koreader/progress (immediate, 202)	sync_status, books_synced, book_results[]?{sha256, book_uuid, synced}, conflicts[]?{book_uuid, conflict_type, device_progress, server_progress, resolution}, timestamp, device_updated
KOReaderProgressPull	GET /api/sync/koreader/metadata/:uuid	uuid, sha256?, title, authors, progress{percentage + optional character/epubcfi/koreader_xpointer/chapter/chapter_progress/page/total_pages}, annotations{highlights?, notes?, bookmarks?, deleted_highlights?, deleted_bookmarks?}, last_sync
KOReaderLibrary	GET /api/sync/koreader/library	library_sync[]{uuid, sha256?, title, author, content_type, percent_read, pages_remaining?, bookmark_count, last_modified}, total_books, last_sync
KOReaderBookmarksPush	POST /api/sync/koreader/bookmarks	sync_status, bookmarks_synced, notes_synced, highlights_synced, total_synced, timestamp
KoboInitialization	GET /api/sync/kobo/:token/v1/initialization	library_sync[]{ContentId, ContentType, Title, Author, PercentRead, PagesRemaining?, BookmarkCount, LastModified, EntitlementId, Revision, MimeType, FileSize, Categories?, BookhoardUUID?}, total_books, last_sync
KoboMarkupSync	POST /api/sync/kobo/:token/markup	Status, MarkupsSynced, BookmarksSynced, DeletedAnnotations?
KoboSyncFromServer	POST /api/sync/kobo/:token/sync-from-server	BooksSynced, BookmarksSent, HighlightsSent
CollectionsJSON	GET /api/collections/:id (all 4 branches)	envelope id, user_id, name, description, color, icon, auto_assign_rules, view_settings, books[], book_count, is_smart, created_at; books = BookInfo
	POST /api/collections/preview	bare {items: BookInfo[]}
SeriesJSON	GET /api/series/books?name=&library_id=	{name, library_id, books: BookInfo[], total}
SearchJSON	GET /api/media-items/search?q=&library_id=	top-level array of SearchMediaItemsUnifiedRow (47 keys)
DashboardSections	GET /api/dashboard/sections	{sections[]: {id, collection_id, is_system, title, description, icon, items: BookInfo[], view_all_url, priority}}
MediaListBothBranches	GET /api/media-items (unscoped)	{data: [38-key resolved map]} (intentionally divergent from the scoped branch until the row-shape collapse batch)
	GET /api/media-items?library_id=&sort=	{data: [raw ListMediaItemsSortedRow, 70 sqlc keys]}
SSRPages	/bookshelf, /collections, /collections/:id, /series, /series/detail, /tags/detail	200 + stable title/substring markers

BookInfo (every card surface): media_item_id, title, author, cover_image_path, has_conflict, format_group, badge.

Changelog (additive keys per batch)

• H0 (initial pin) — the shapes above, captured on the unmodified tree at commit b4ae423.
• H1 (progress canonicalization) — GET /api/media-items/:id/progress Whitelisted ADDITIVE keys (fields the GetUniversalProgressRow already carries, now surfaced by the shared row serializer): audio_offset_seconds, reading_mode, viewport_x, viewport_y, zoom_level, scroll_position_x, scroll_position_y, panel_number. No key removed or renamed; the no-rows and PUT-response shapes are untouched. The web PUT binds SaveProgressRequest directly — its parse set for existing clients is IDENTICAL to the old DTO (verified against web/src reader.ts:3113-3143 and audio.ts:518-520); keys the web never sent (viewport_x/y, panel_number) merely become honor-able. All other pins unchanged.
• H3-prep (web annotation CRUD freeze) — NEW pinned surfaces, captured pre-refactor per the pin-first doctrine: POST/PUT/DELETE /api/media-items/:id/{notes,highlights,bookmarks} — create 201 / update 200 / delete 204, response bodies are the full sqlc row models (MediaNotes 19 keys, MediaBookmarks 20 keys, MediaHighlights 27 keys — listed in contract_pins_test.go). No keys removed; nothing was pinned here before. Capturing these immediately caught a LIVE server bug: applyNoteLWW omitted UserID from its UpdateMediaNoteForSync params, so every dedup-hit note re-save (same content+position ⇒ same key) returned 500 ("update note: no rows in result set") — the LWW update could never land. Fixed in the same commit (server-side fix, freely allowed); the pins are its regression net.
• H3 (PUT fixes routed through the sync service) — PUT /api/media-items/:id/bookmarks/:bookmarkId now parses ADDITIONAL optional request keys mirroring its create sibling: cfi_position, page_number, chapter_number, percentage, chapter_reference (all zero-value-preserving: omitted keys coalesce with the stored row inside the service, so legacy clients that send only title/notes/position keep every stored locator — verified by TestH3_BookmarkPUTPreservesLocators). The PUT lands as an id-targeted LWW edit under the row's stored dedup key (BookmarkID/NoteID mirror SaveHighlightRequest.HighlightID), so last_modified_source / last_modified_at are now maintained. Note PUT unchanged on the wire (content+position) but routed through the service the same way. Response shapes unchanged (same row models — pins green).
• H4 (unified row completion) — /api/media-items/search rows[] whitelisted ADDITIVE keys abridged, audio_metadata: the unified search row gained the two columns it lagged (EXPLAIN plan-identical — both ride the already-scanned media_items side), and the endpoint serializes the row directly, so they emit. No key removed or renamed.
• Origin writer attribution (post-hardening parity) — every web-API write endpoint now parses an ADDITIVE optional request key origin ("android" | "ios"; anything else stays "web") and records it as the writer: progress PUT → last_sync_source, bookmark/note/highlight create+PUT → last_modified_source (and origin_source on bookmark creates, now normalized through the same gate). Precedence on progress: a paired X-Device-Token still outranks origin. The web SPA sends no origin — its rows are unchanged. Request-parse widening documented per rule 2; response shapes untouched; locked by TestOriginWriterAttribution.

Changelog entry — multi-file audiobooks (2026-10-18)

• GET /api/media-items/:id/chapters gained ADDITIVE response keys layout ("" for single-file books, "folder" or "zip" for multi-file books) and tracks[] (index, url, title, path, start, end, duration_seconds, size_bytes; empty for single-file books). Existing keys (media_item_id, chapters, duration_seconds, codec_profile) unchanged — no pin existed on this endpoint, so no pin widened; recorded here per rule 2's spirit.
• New sibling endpoint GET /api/media-items/:id/audio/:trackIndex streams one track of a multi-file book (Range-capable). New surface, not a modification; gated by library visibility like ServeFile (403 on hidden libraries).
• media_items.audio_metadata payload gained additive JSON keys layout and tracks (see schema.sql comment). Consumers parse additively; stored single-file payloads are unchanged.

2026-10-19 — A2b: metadata array/model overhaul (phase 1)

• media_items.series_number widened INTEGER → DOUBLE PRECISION across all surfaces. JSON serializers emit the same key; values are JSON numbers, so integer-valued ordinals are byte-identical ("3") and decimal ordinals (0.5/1.5) now serialize as decimals instead of truncating. Clients reading JSON numbers are unaffected; the Android client parses either form.
• GET /api/media-items* / detail responses gain ADDITIVE keys: subtitle (string), explicit (bool), authors ([]string, write truth — legacy author display string still emitted, now trigger-regenerated), publishers ([]string; legacy publisher likewise).
• POST/PUT /api/media-items* accept ADDITIVE request keys subtitle, explicit, authors, publishers; legacy author/publisher singles still accepted (arrays win when both present).
• Override keys unchanged: author/publisher override names are preserved for existing rows and now protect the arrays.

2026-10-19 — A2c: collections/playlists phase

• GET /api/collections/:id gained ADDITIVE up_next (array of {id, title, author, cover, format, progress} — the static collection's reading queue head in position order, viewer-progress-aware; empty array for smart collections).
• GET /api/dashboard/sections rows gained ADDITIVE is_playlist (bool — collection was imported from a sidecar ordering construct) and complete (bool — hide_finished filtered every item out of a non-empty collection; render "playlist complete").
• PUT /api/collections/:id/items/order is a NEW endpoint (owner-only, full ordered id list → positions 1..n).
• User collections are dashboard-eligible by default (the show_on_dashboard gate is gone from GetUserCollectionsForDashboard); hiding stays dashboard-prefs (hidden_collections) territory.
• GET /api/media-items (unscoped resolved-map branch) and GET /api/media-items/:id gained the same ADDITIVE keys as the raw branch: subtitle, explicit, authors, publishers.
• GET /api/media-items/search rows gained the same ADDITIVE keys plus the search pairs authors_search, publishers_search, narrators_search (the unified SELECT now carries the new columns).
• author_filter / publisher_filter are now per-element against the arrays; a comma-joined value is OR-within-facet multi-select. The param NAMES and single-value behavior are unchanged.

2026-10-19 — Faceted navigation Batch A: new filter facets

• GET /api/media-items/search accepts ADDITIVE REQUEST params (query string, response shapes unchanged): story_arc_filter, imprint_filter, edition_filter (scalar-fuzzy), age_rating_filter, format_group_filter, manga_type_filter, reading_direction_filter (exact), explicit_filter, abridged_filter (bool: absent = unset, "true"/"false" = bound). A comma-joined value is OR-within-facet multi-select on every new TEXT facet (the SQL splits via string_to_array). services/facet_registry.go is the Go-side source of truth; the drift-guard unit test (TestFacetRegistrySQLDrift) pins registry ↔ SQL agreement.
• Genre model v2 changes SCANNER OUTPUT (not a wire contract): sidecar genre selection now stores only top-level shelves in genre (subgenres become tags mapped to their parent shelf; unknown-only input leaves genre empty). Existing rows are unaffected until a rescan/re-extract.

2026-10-19 — Faceted navigation Batch B/C: chips, FacetLink, facets endpoint

• GET /api/media-items/facets?facet=X&library_id=... is a NEW pinned surface (registry-driven value+count listing for the filter panel): 200 → {facet, values[]: {value, display?, count}} (display present only for language codes → names); 400 for unknown facets and for bool/range facets (they render as toggles, not lists); 403 for an explicit library_id outside the requester's visible set.
• GET /api/media-items/search (JSON) and its HTMX grid partial now share ONE param parser (handlers.ParseMediaSearchParams). Behavior notes: the HTMX branch previously dropped publisher_filter and narrator_filter silently — it now honors every facet param; and an explicit hidden-library library_id is now a 403 on BOTH branches (was: silent empty set on JSON). Response shapes are untouched; the search row pin (47 keys) is unchanged.
• language_filter now normalizes comma elements through the ingest language map, so display names ("English") match rows stored as ISO codes ("en"). The param name is unchanged; single-code values behave identically.
• Web: the bookshelf gains an active-filter chips bar (client-local form state rendered by Alpine; templ owns the markup) and applyURLFilters accepts the new facet params.

2026-10-19 — Scanner Phase 4: manifest-gated sidecar detection

• No wire change: media_scan_state lives in its own table, so the raw-row list JSON cannot contain scanner state (guarded by an assertion in TestScanManifestLifecycle).
• Scanner BEHAVIOR: size-stable books are now checked against a stored sidecar fingerprint ({"v":1,"entries":[{n,s,m-nanos,h-sha256},...]}). Stat-pair-identical → skip with zero reads; stat drift → hash reconciliation (equal hash = host repoint, stat pair refreshed, no re-extraction; differing hash = metadata re-extract + fresh manifest, metadata_overrides protected, playlist collections synced on the change path — a "(Reading Order)" entry added to an EXISTING book now creates its collection). NULL stored manifest → backfill only, no re-extraction (deploy-day rule). Multifile folder/zip books keep the Phase 3 behavior (their change path does not consume sidecar extractors yet — gating them would flip books "changed" for nothing).

2026-10-19 — Scanner Phase 5: quiet scans, coalescing, watcher probe

• POST /api/scanner/scan now answers 409 (with active_job_id) when a scan for the same library is already pending/running — coalescing instead of stacking a second full scan. The endpoint was never pinned; no existing response shape changed. Poll ticks and watcher events coalesce silently (no second job enqueued).
• Scan jobs carry a trigger label (poll/watcher/admin/initial); the per-scan log is now ONE summary line: Scan of <folders> (trigger: <t>): N files, N changed, N new, Xs, N errors. Per-file "processing/success/skipping" chatter is gone; inserts, changes, and errors still log individually.
• JobResult gains additive job_type (coalescing needs it; the scan-progress surfaces tolerate unknown keys).
• Watcher: a failed capability probe (one Add on a root folder) disables the watcher with ONE startup line instead of per-directory warning spam.

2026-10-20 — Stale pin repair (behavior shipped in 7e586df4)

• GET /api/media-items/search with an explicit library_id outside the requesting user's visible set — INCLUDING a nonexistent id — answers 403 (library not accessible), not 404. The family-lens standing rule (AGENTS.md) mandates 403 for every out-of-set id so library existence is not enumerable; Phase 1 (7e586df4) shipped it and documented it here, but the invalid library_id subtest in search_test.go still pinned the pre-family-lens 404. Pin widened to the shipped 403; no server change (discovered during the Phase 6 full-suite run; the Phase 6 diff touches no handler code).
• TestWebSocketPingPong DELETED: it asserted a text-level {"type":"ping"} → {"type":"pong"} exchange that no server code has ever implemented and no client (web/src, Android) has ever sent. It passed vacuously through its read-timeout path (deadline error skipped the asserts) until interleaved scan_progress broadcasts from the live server's scans started reaching the socket first. Keepalive is the standard protocol-level ping the hub already sends. Never-built behavior is not a pinnable contract.

2026-10-20 — Scanner Phase 7: all-facets shape + SSR facet panel

• GET /api/media-items/facets gains a SECOND shape: with NO facet param it answers {"facets": {<param>: [{"value","display?","count"}, …]}} — every listable facet in one response, counts computed under the ACTIVE FILTERS (the same query params the search endpoint takes; each facet is counted under the others — its own filter does not restrict its own list). library_id is optional on this shape: absent = the All-Libraries view. Family-lens unchanged: an explicit out-of-set library_id is 403 on both shapes.
• The PINNED single-facet shape (?facet=X&library_id=… → {facet, values[]}) is byte-identical in semantics; its implementation now routes through the same UNION-ALL query with neutral filters (the 14 per-column SQL queries were deleted — no wire change).
• NEW SSR partial: GET /media-items/facet-panel (session-cookie auth, bookshelf-protected group) renders the drawer's value lists — templ-rendered, registry order, library-type-scoped sections, active values marked. An out-of-set explicit library_id is 403 HTML.
• The 14 per-column facet SQL queries are replaced by one FacetCountsUnderFilters UNION-ALL statement; the registry drift guard now reflects over both generated param structs.

2026-10-20 — Filter plumbing: facet-panel partial removed; field-values library_id optional

• GET /media-items/facet-panel (the SSR partial documented above, same day) is REMOVED — the panel UX was rejected on scale grounds and the filter drawer becomes a rule builder with no other mode. Never contract-pinned while it lived (same-day entry, same-day removal); TestFacetPanelPartial was deleted with the behavior. The all-facets JSON endpoint and its two shapes are UNCHANGED and stay pinned; FacetCountsUnderFilters and the registry drift guard keep their roles (the engine outlives the panel).
• GET /api/media-items/search?field=value autocomplete (author/genre/tags/narrator/series/language): library_id is now OPTIONAL — absent means every library VISIBLE to the caller (the per-user family lens applies inside the query), fixing the All-Books view's "library_id is required" 400. An explicit out-of-set library_id is now a 403 on this endpoint too (was: silent empty result for hidden libraries — the silent pass is the anti-pattern the standing rule forbids). The single-facet /api/media-items/facets?facet=X shape still REQUIRES library_id — that pin is unchanged.

2026-10-20 — Filter rule builder: the qf wire param + composed engines (faceted plan Phase 9)

• NEW wire param on the search endpoint family: qf — a JSON rule tree {mode: all|any, children: [{field, op, values} | {mode, children}]}. Fields/operators are registry-typed constants (services/filter_registry.go — ~55 fields incl. every book column, file/lifecycle facts, the three HOLDING STRUCTURES — library, collections (playlists included, excluded IS NOT TRUE membership), series (primary + alternate_info full membership) — and the whole user-relative class reading the VIEWER's rows). Unknown fields or operators, disallowed operator/field pairs, malformed values, empty groups, and nesting beyond 8 are a 400 (ErrInvalidFilterTree), never a silently-ignored token. Values are always bound pgx parameters — an injection-shaped value is just a value.
• Precedence (pinned): when a request carries BOTH qf and flat facet params, qf wins — the flat params are ignored by the engine. Flat links keep working unchanged server-side (and a tree equivalent to a flat link returns identical rows — pinned by test).
• The composed engines (services/filter_query.go) run the mandatory WHERE head (visibility join + archived + missing-scan — family lens) for search, suggestions, and count alike; row shape is identical to SearchMediaItemsUnifiedRow. A rule naming a hidden library matches nothing; an explicit out-of-set library_id stays a 403 on every entry point.
• NEW endpoint GET /api/media-items/filter-count?qf=…[&library_id=…] — the builder's live count: a true COUNT(*) of the rules + scope (not the page length). Family-lens 403 applies.
• Field-values autocomplete UPGRADED in place (same route): ANY registry field key in its bare form (?note_text=…, ?collection=…, ?library=…) routes to the field-values path; with a qf param the OTHER rules bind as filters (symmetric under-filter suggestions, zero-match values never offered). Without a tree, the classic six (author/genre/tags/narrator/series/language) keep their static queries and response shape — pinned contract, unchanged. ID-valued suggestions (library/collection/added_by) add an OPTIONAL display field to result items (value stays the id; omitempty elsewhere).
• The Phase-7 facet-panel partial stays removed; the all-facets JSON endpoint and its two shapes are unchanged (they serve flat params — the builder's UI does not use them).

2026-10-20 — Scanner Phase 10: extractor version (ev) mechanism

• media_scan_state.manifest JSONB gains the reserved ev key: the parser version that produced the item's CURRENT row. ABSENT means "v1-or-unknown" — decode normalizes it to 1.
• NEW setting scan_extractor_version (int, default 1, min 1, category scanner): the unified GET/PUT /api/system/settings list now includes this key (additive; the list was not a pinned closed set). Writes stay settings-API-only — direct SQL leaves the registry cache stale (known gotcha).
• Scanner behavior: a stored manifest whose ev ≠ the setting is treated as CHANGED → full re-extract + the fresh manifest stamps the new ev. Stamping is TRUTHFUL-ONLY: create paths and post-extraction stores stamp; deploy-day BACKFILL and the extract-failure guard store UNSET (the row never claims a version its data can't prove — a mid-backfill bump self-heals next poll). Repoint reconciliation preserves the stored ev (stat-only refresh, nothing re-extracted).
• Catch-up is BOUNDED per scan pass (evReextractBudgetPerScan = 100, atomic, reset at each ScanFolders): resumable across polls, no force rescan, no flood. Budget exhaustion leaves ev stale → later polls resume.
• Deploy-day rule (unchanged, re-verified live): manifests without ev ride v1; at default setting 1 the first post-release scan re-extracts NOTHING (0 changed on 1198 files).

2026-10-20 — Scanner Phase 11: the processing-issues ledger

• processing_issues.media_item_id is now NULLABLE and the table gains file_path (TEXT, deliberately unbounded — an over-long path is itself a failure mode the ledger must survive): ROW-LESS import failures (no media_items row exists) key on (library, file_path); item-backed rows keep (item, type). The widened dedup is the COALESCE-expression unique index idx_processing_issues_dedup; the old table-level UNIQUE stays (row-less rows have NULL items, pairwise distinct under unique constraints).
• GET /api/libraries/:id/issues/list — the media_items join is now LEFT: row-less issues APPEAR (they were unrepresentable before). Response shape unchanged for item-backed rows; row-less rows carry media_item_id: "" and a title fallen back to the file path.
• FAMILY LENS now applies to ALL issues endpoints (list, stats, resolve, delete) and the SSR page — they leak media metadata. An explicit library id outside the requester's visible set is a 403 (this tightens one pre-lens behavior: a nonexistent library id used to return 200-empty; it is trivially out-of-set → 403 now, pinned).
• NEW scanner wirings: CreateMediaItem failure → row-less import_failed (error); extraction hard-fail with fallback import → item-backed metadata_extraction_failed (warning); change-path re-extraction failure → item-backed metadata_extraction_failed.
• Auto-resolution: a successful import resolves row-less issues by (library, file_path); a successful (re)extraction resolves the item's open issues. Unchanged-skip scans do NOT resolve (extraction never re-ran — the degraded metadata is still degraded). A repeat failure reopens a resolved row (upsert sets resolved=false).