Files
bookhoard/docs/contract_pins.md
T
John O'Keefe 6216bd9bcf feat(audiobooks): multi-file books (folder-of-tracks + zip) + exhaustive metadata ingestion
Multi-file book ingest: a directory of ≥2 audio files (recursively —
disc-part folders like CD1/Disc 2 merge upward, siblings never merge)
ingests as ONE book whose file_path is the book-root directory; a zip
of audio tracks is its own book (file_path = the archive). Both write
the documented chapters abstraction {index,title,start,end} into
audio_metadata and add additive keys: layout ("folder"|"zip") and a
tracks[] manifest (per-track path, global-seconds start/end, duration,
size). Pre-grouping one-track strays under a book root are absorbed on
rescan (annotations cascade). Rescans take a stat-only fast path
(track count + total size + known durations unchanged → no re-probe —
a 1189-track book must not re-ffprobe every auto-scan); a zero-duration
book counts as changed so a ffprobe-equipped rescan fills durations in.
Folder-book roots count as present in the archive lifecycle pass, and
per-item rescan (RescanMediaItem) routes multi-file books through a
book-level refresh — the generic updater would read the root/zip as if
it were the book file and clobber format_group with "unknown".

Serving: GET /api/media-items/:id/chapters gains additive layout +
tracks keys (urls → the new endpoint). New GET
/api/media-items/:id/audio/:trackIndex streams one track — folder
tracks served from the real file (full Range via ServeContent), zip
tracks via a reopen-and-skip ReadSeeker over the entry (deflate has no
random access; acceptable at audio bitrates; stored entries stay
section readers). Both endpoints gate on library visibility (403 on
hidden libraries — family-lens rule). ServeFile and OPDS DownloadBook
refuse directory paths (folder books have no single-file download).

Web player (next commit) consumes the manifest; progress/bookmarks/
deep-links keep their global-seconds coordinates, so the server-side
progress model is untouched. sync learns FormatGroupAudiobook
(percentage passthrough in ConvertProgress); detectFormatType stops
reporting "unknown" for audio.

Metadata ingestion (Batch B) — read everything available, degrade
gracefully, documented precedence sidecars → folder structure →
embedded tags → filename: ffprobe tags are now a case-insensitive
lookup map (freeform keys pass through verbatim); composer fills
narrators (Audible's own convention, kept as ONE string — no reliable
multi-narrator delimiter); grouping + Audible freeform atoms
(SERIES/SERIES-PART/SUBTITLE/ASIN/ABRIDGED) fill series/index/
subtitle-join/ASIN/abridged (explicit sidecar abridged-false is
guarded); date/genre fall back to tags; album/album_artist/artist
fill title/author. Genre persists at create for all formats.
Embedded cover art (mp3 APIC, m4b covr, flac picture) extracts via one
optional ffmpeg call as a "<book>.cover.jpg" sibling — sidecar covers
win. CUE sheets (FILE-matched, INDEX 01 at 75fps) are the lowest-
priority chapter source under sidecars and embedded m4b chapters. A
lone m3u/m3u8 in a book root imposes playlist ordering + #EXTINF
names; (disc, track) tags reorder tracks when coverage is complete.
extractMetadata's default branch no longer pre-fills the filename as
title (tags must outrank it) — final filename fallbacks now exist on
BOTH the create and rescan paths (TestRescanKeepsFilenameTitle pins
that rescans cannot blank a title); tagless zip books take the zip's
own name, not their parent folder's.

Accepted containers widen to .m4b .m4a .mp3 .flac .zip .opus .ogg .oga
.aac .wav with real media types registered (audio/ogg per RFC 7845 for
Opus-in-Ogg) — all browser-playable; .wma deliberately excluded.

Tests: unit tests for CUE parsing, playlist hints, tag fallbacks
(synthetic ffprobe reports — ffmpeg cannot write Audible freeform
atoms), ordinal parsing, tag ordering, natural sort, grouping rules;
integration suite (11 tests) over the live dev DB: folder/zip ingest +
idempotency, stray absorption, chapters manifest, per-track streaming
with Range, hidden-library 403s, embedded covers + sidecar-wins, CUE
chapters, embedded m4b chapters, playlist ordering, tag fallbacks, all
five new containers with content-type assertions, rescan title pin.
Fixture media is real committed-format binary content living untracked
under internal/services/testdata/ (gitignored by policy; recipes in
docs/contributing/development.md → Media fixtures). Test image now
ships ffmpeg to match the runtime's audio toolchain.
2026-10-19 07:54:28 -04:00

8.9 KiB

Wire-contract pins (hardening batches)

These are the response SHAPES (JSON key sets, not values) frozen by the TestContractPin_* suite in cmd/server/tests/contract_pins_test.go (batch H0). The plan they serve is ~/.opencode/plan/hardening-canonical-types.md.

Rules

  1. Frozen: no existing response key may be RENAMED or REMOVED on any endpoint below (JSON or KOReader/Kobo adapters). Requests are frozen too — no parsing changes.
  2. Additive keys are allowed and expected — but a batch must widen the allowed slice in the test AND record the addition in the changelog below, in the same commit.
  3. KOReader / Kobo / progress surfaces are the hard freeze: the KOReader plugin is an external client whose handlers keep response shapes byte-identical (only their internals may route through shared funnels).
  4. A behavior fix that would require narrowing or REMOVING a pinned key needs the user's explicit sign-off before the pin is touched — a pin freezing buggy behavior is a known-bugs ledger, not a license.

Pinned surfaces

Test Surface Shape
ProgressWeb GET /api/media-items/:id/progress (percentage-only write) flat map: id, media_item_id, user_id, current_page, total_pages, last_read_at, percentage, character_offset, epubcfi, chapter, chapter_progress, format_group, total_characters, chapter_count, last_sync_device, last_sync_source, last_sync_timestamp
… conditional bolt-ons context_text (when stored valid); css_selector, anchor_href, char_offset, rewritten epubcfi (when the anchor verify path fires)
PUT /api/media-items/:id/progress response database.ReadingProgress JSON (25 keys incl. audio_offset_seconds, viewport/zoom/scroll/panel/reading_mode)
GET with no rows exactly {current_page, total_pages}
KOReaderProgressPush POST /api/sync/koreader/progress (immediate, 202) sync_status, books_synced, book_results[]?{sha256, book_uuid, synced}, conflicts[]?{book_uuid, conflict_type, device_progress, server_progress, resolution}, timestamp, device_updated
KOReaderProgressPull GET /api/sync/koreader/metadata/:uuid uuid, sha256?, title, authors, progress{percentage + optional character/epubcfi/koreader_xpointer/chapter/chapter_progress/page/total_pages}, annotations{highlights?, notes?, bookmarks?, deleted_highlights?, deleted_bookmarks?}, last_sync
KOReaderLibrary GET /api/sync/koreader/library library_sync[]{uuid, sha256?, title, author, content_type, percent_read, pages_remaining?, bookmark_count, last_modified}, total_books, last_sync
KOReaderBookmarksPush POST /api/sync/koreader/bookmarks sync_status, bookmarks_synced, notes_synced, highlights_synced, total_synced, timestamp
KoboInitialization GET /api/sync/kobo/:token/v1/initialization library_sync[]{ContentId, ContentType, Title, Author, PercentRead, PagesRemaining?, BookmarkCount, LastModified, EntitlementId, Revision, MimeType, FileSize, Categories?, BookhoardUUID?}, total_books, last_sync
KoboMarkupSync POST /api/sync/kobo/:token/markup Status, MarkupsSynced, BookmarksSynced, DeletedAnnotations?
KoboSyncFromServer POST /api/sync/kobo/:token/sync-from-server BooksSynced, BookmarksSent, HighlightsSent
CollectionsJSON GET /api/collections/:id (all 4 branches) envelope id, user_id, name, description, color, icon, auto_assign_rules, view_settings, books[], book_count, is_smart, created_at; books = BookInfo
POST /api/collections/preview bare {items: BookInfo[]}
SeriesJSON GET /api/series/books?name=&library_id= {name, library_id, books: BookInfo[], total}
SearchJSON GET /api/media-items/search?q=&library_id= top-level array of SearchMediaItemsUnifiedRow (47 keys)
DashboardSections GET /api/dashboard/sections {sections[]: {id, collection_id, is_system, title, description, icon, items: BookInfo[], view_all_url, priority}}
MediaListBothBranches GET /api/media-items (unscoped) {data: [38-key resolved map]} (intentionally divergent from the scoped branch until the row-shape collapse batch)
GET /api/media-items?library_id=&sort= {data: [raw ListMediaItemsSortedRow, 70 sqlc keys]}
SSRPages /bookshelf, /collections, /collections/:id, /series, /series/detail, /tags/detail 200 + stable title/substring markers

BookInfo (every card surface): media_item_id, title, author, cover_image_path, has_conflict, format_group, badge.

Changelog (additive keys per batch)

  • H0 (initial pin) — the shapes above, captured on the unmodified tree at commit b4ae423.
  • H1 (progress canonicalization) — GET /api/media-items/:id/progress Whitelisted ADDITIVE keys (fields the GetUniversalProgressRow already carries, now surfaced by the shared row serializer): audio_offset_seconds, reading_mode, viewport_x, viewport_y, zoom_level, scroll_position_x, scroll_position_y, panel_number. No key removed or renamed; the no-rows and PUT-response shapes are untouched. The web PUT binds SaveProgressRequest directly — its parse set for existing clients is IDENTICAL to the old DTO (verified against web/src reader.ts:3113-3143 and audio.ts:518-520); keys the web never sent (viewport_x/y, panel_number) merely become honor-able. All other pins unchanged.
  • H3-prep (web annotation CRUD freeze) — NEW pinned surfaces, captured pre-refactor per the pin-first doctrine: POST/PUT/DELETE /api/media-items/:id/{notes,highlights,bookmarks} — create 201 / update 200 / delete 204, response bodies are the full sqlc row models (MediaNotes 19 keys, MediaBookmarks 20 keys, MediaHighlights 27 keys — listed in contract_pins_test.go). No keys removed; nothing was pinned here before. Capturing these immediately caught a LIVE server bug: applyNoteLWW omitted UserID from its UpdateMediaNoteForSync params, so every dedup-hit note re-save (same content+position ⇒ same key) returned 500 ("update note: no rows in result set") — the LWW update could never land. Fixed in the same commit (server-side fix, freely allowed); the pins are its regression net.
  • H3 (PUT fixes routed through the sync service) — PUT /api/media-items/:id/bookmarks/:bookmarkId now parses ADDITIONAL optional request keys mirroring its create sibling: cfi_position, page_number, chapter_number, percentage, chapter_reference (all zero-value-preserving: omitted keys coalesce with the stored row inside the service, so legacy clients that send only title/notes/position keep every stored locator — verified by TestH3_BookmarkPUTPreservesLocators). The PUT lands as an id-targeted LWW edit under the row's stored dedup key (BookmarkID/NoteID mirror SaveHighlightRequest.HighlightID), so last_modified_source / last_modified_at are now maintained. Note PUT unchanged on the wire (content+position) but routed through the service the same way. Response shapes unchanged (same row models — pins green).
  • H4 (unified row completion) — /api/media-items/search rows[] whitelisted ADDITIVE keys abridged, audio_metadata: the unified search row gained the two columns it lagged (EXPLAIN plan-identical — both ride the already-scanned media_items side), and the endpoint serializes the row directly, so they emit. No key removed or renamed.
  • Origin writer attribution (post-hardening parity) — every web-API write endpoint now parses an ADDITIVE optional request key origin ("android" | "ios"; anything else stays "web") and records it as the writer: progress PUT → last_sync_source, bookmark/note/highlight create+PUT → last_modified_source (and origin_source on bookmark creates, now normalized through the same gate). Precedence on progress: a paired X-Device-Token still outranks origin. The web SPA sends no origin — its rows are unchanged. Request-parse widening documented per rule 2; response shapes untouched; locked by TestOriginWriterAttribution.

Changelog entry — multi-file audiobooks (2026-10-18)

  • GET /api/media-items/:id/chapters gained ADDITIVE response keys layout ("" for single-file books, "folder" or "zip" for multi-file books) and tracks[] (index, url, title, path, start, end, duration_seconds, size_bytes; empty for single-file books). Existing keys (media_item_id, chapters, duration_seconds, codec_profile) unchanged — no pin existed on this endpoint, so no pin widened; recorded here per rule 2's spirit.
  • New sibling endpoint GET /api/media-items/:id/audio/:trackIndex streams one track of a multi-file book (Range-capable). New surface, not a modification; gated by library visibility like ServeFile (403 on hidden libraries).
  • media_items.audio_metadata payload gained additive JSON keys layout and tracks (see schema.sql comment). Consumers parse additively; stored single-file payloads are unchanged.