Phase 1 of the series/collections/metadata overhaul
(~/.opencode/plan/series-collections-metadata-overhaul.md).
Schema (in place, idempotent):
- media_items.series_number INTEGER -> DOUBLE PRECISION: real-world
ordinals are decimals (0.5, 1.5 novellas); INTEGER silently truncated
them (calibre 2.5 pinned as 2 in tests).
- + subtitle TEXT, explicit BOOLEAN (sidecar ingest; ABS explicit /
Audible isAdult).
- + authors/authors_search/publishers/publishers_search TEXT[] NOT NULL
(narrators A2a precedent). Arrays are write truth; author/publisher
varchars become trigger-regenerated display strings
(sync_author_publisher_display) so every varchar consumer (trgm
search, facets, API, OPDS/Kobo) keeps working unchanged.
- Row-level NULL normalization for the new arrays in
set_library_type_name (A2 lesson: Go nil -> explicit NULL -> 23502).
- Backfills ARRAY[author]/ARRAY[publisher] — deliberately no
comma-splitting ('Tolkien, J.R.R.' would corrupt; scanner kept only
authors[0] historically).
- collection_items.position INTEGER + (collection_id, position) index
for playlist ordering.
Go: sqlc regen; MediaMetadata/handler params gain Subtitle/Explicit/
Authors/Publishers (legacy singles still accepted, arrays win);
SeriesNumber int32 -> float64 end to end; metadata_overrides adapted
with legacy override keys ('author'/'publisher') preserved so existing
user overrides survive; bulk author edit writes the array.
Pins widened (docs/contract_pins.md): raw sqlc row key set + the six
new columns; changelog entry added.
Verified: unit suite green; integration sweep (bulk/smart collections/
scanner/stats/sync/queue/calibre/pins/audiobooks) green against the
shared dev DB after in-place migration. Pre-existing unrelated red:
TestOwnershipAndAdminTestSuite/TestAdminCreateUser timezone inheritance
fails identically at the pre-change commit (live shared state).
9.9 KiB
9.9 KiB
Wire-contract pins (hardening batches)
These are the response SHAPES (JSON key sets, not values) frozen by the
TestContractPin_* suite in cmd/server/tests/contract_pins_test.go
(batch H0). The plan they serve is ~/.opencode/plan/hardening-canonical-types.md.
Rules
- Frozen: no existing response key may be RENAMED or REMOVED on any endpoint below (JSON or KOReader/Kobo adapters). Requests are frozen too — no parsing changes.
- Additive keys are allowed and expected — but a batch must widen the
allowedslice in the test AND record the addition in the changelog below, in the same commit. - KOReader / Kobo / progress surfaces are the hard freeze: the KOReader plugin is an external client whose handlers keep response shapes byte-identical (only their internals may route through shared funnels).
- A behavior fix that would require narrowing or REMOVING a pinned key needs the user's explicit sign-off before the pin is touched — a pin freezing buggy behavior is a known-bugs ledger, not a license.
Pinned surfaces
| Test | Surface | Shape |
|---|---|---|
ProgressWeb |
GET /api/media-items/:id/progress (percentage-only write) |
flat map: id, media_item_id, user_id, current_page, total_pages, last_read_at, percentage, character_offset, epubcfi, chapter, chapter_progress, format_group, total_characters, chapter_count, last_sync_device, last_sync_source, last_sync_timestamp |
| … conditional bolt-ons | context_text (when stored valid); css_selector, anchor_href, char_offset, rewritten epubcfi (when the anchor verify path fires) |
|
PUT /api/media-items/:id/progress response |
database.ReadingProgress JSON (25 keys incl. audio_offset_seconds, viewport/zoom/scroll/panel/reading_mode) |
|
GET with no rows |
exactly {current_page, total_pages} |
|
KOReaderProgressPush |
POST /api/sync/koreader/progress (immediate, 202) |
sync_status, books_synced, book_results[]?{sha256, book_uuid, synced}, conflicts[]?{book_uuid, conflict_type, device_progress, server_progress, resolution}, timestamp, device_updated |
KOReaderProgressPull |
GET /api/sync/koreader/metadata/:uuid |
uuid, sha256?, title, authors, progress{percentage + optional character/epubcfi/koreader_xpointer/chapter/chapter_progress/page/total_pages}, annotations{highlights?, notes?, bookmarks?, deleted_highlights?, deleted_bookmarks?}, last_sync |
KOReaderLibrary |
GET /api/sync/koreader/library |
library_sync[]{uuid, sha256?, title, author, content_type, percent_read, pages_remaining?, bookmark_count, last_modified}, total_books, last_sync |
KOReaderBookmarksPush |
POST /api/sync/koreader/bookmarks |
sync_status, bookmarks_synced, notes_synced, highlights_synced, total_synced, timestamp |
KoboInitialization |
GET /api/sync/kobo/:token/v1/initialization |
library_sync[]{ContentId, ContentType, Title, Author, PercentRead, PagesRemaining?, BookmarkCount, LastModified, EntitlementId, Revision, MimeType, FileSize, Categories?, BookhoardUUID?}, total_books, last_sync |
KoboMarkupSync |
POST /api/sync/kobo/:token/markup |
Status, MarkupsSynced, BookmarksSynced, DeletedAnnotations? |
KoboSyncFromServer |
POST /api/sync/kobo/:token/sync-from-server |
BooksSynced, BookmarksSent, HighlightsSent |
CollectionsJSON |
GET /api/collections/:id (all 4 branches) |
envelope id, user_id, name, description, color, icon, auto_assign_rules, view_settings, books[], book_count, is_smart, created_at; books = BookInfo |
POST /api/collections/preview |
bare {items: BookInfo[]} |
|
SeriesJSON |
GET /api/series/books?name=&library_id= |
{name, library_id, books: BookInfo[], total} |
SearchJSON |
GET /api/media-items/search?q=&library_id= |
top-level array of SearchMediaItemsUnifiedRow (47 keys) |
DashboardSections |
GET /api/dashboard/sections |
{sections[]: {id, collection_id, is_system, title, description, icon, items: BookInfo[], view_all_url, priority}} |
MediaListBothBranches |
GET /api/media-items (unscoped) |
{data: [38-key resolved map]} (intentionally divergent from the scoped branch until the row-shape collapse batch) |
GET /api/media-items?library_id=&sort= |
{data: [raw ListMediaItemsSortedRow, 70 sqlc keys]} |
|
SSRPages |
/bookshelf, /collections, /collections/:id, /series, /series/detail, /tags/detail |
200 + stable title/substring markers |
BookInfo (every card surface): media_item_id, title, author, cover_image_path, has_conflict, format_group, badge.
Changelog (additive keys per batch)
- H0 (initial pin) — the shapes above, captured on the unmodified tree
at commit
b4ae423. - H1 (progress canonicalization) —
GET /api/media-items/:id/progressWhitelisted ADDITIVE keys (fields theGetUniversalProgressRowalready carries, now surfaced by the shared row serializer):audio_offset_seconds, reading_mode, viewport_x, viewport_y, zoom_level, scroll_position_x, scroll_position_y, panel_number. No key removed or renamed; the no-rows and PUT-response shapes are untouched. The web PUT bindsSaveProgressRequestdirectly — its parse set for existing clients is IDENTICAL to the old DTO (verified against web/src reader.ts:3113-3143 and audio.ts:518-520); keys the web never sent (viewport_x/y, panel_number) merely become honor-able. All other pins unchanged. - H3-prep (web annotation CRUD freeze) — NEW pinned surfaces, captured
pre-refactor per the pin-first doctrine:
POST/PUT/DELETE /api/media-items/:id/{notes,highlights,bookmarks}— create 201 / update 200 / delete 204, response bodies are the full sqlc row models (MediaNotes 19 keys, MediaBookmarks 20 keys, MediaHighlights 27 keys — listed in contract_pins_test.go). No keys removed; nothing was pinned here before. Capturing these immediately caught a LIVE server bug:applyNoteLWWomittedUserIDfrom itsUpdateMediaNoteForSyncparams, so every dedup-hit note re-save (same content+position ⇒ same key) returned 500 ("update note: no rows in result set") — the LWW update could never land. Fixed in the same commit (server-side fix, freely allowed); the pins are its regression net. - H3 (PUT fixes routed through the sync service) —
PUT /api/media-items/:id/bookmarks/:bookmarkIdnow parses ADDITIONAL optional request keys mirroring its create sibling:cfi_position, page_number, chapter_number, percentage, chapter_reference(all zero-value-preserving: omitted keys coalesce with the stored row inside the service, so legacy clients that send only title/notes/position keep every stored locator — verified by TestH3_BookmarkPUTPreservesLocators). The PUT lands as an id-targeted LWW edit under the row's stored dedup key (BookmarkID/NoteID mirror SaveHighlightRequest.HighlightID), solast_modified_source/last_modified_atare now maintained. Note PUT unchanged on the wire (content+position) but routed through the service the same way. Response shapes unchanged (same row models — pins green). - H4 (unified row completion) —
/api/media-items/searchrows[] whitelisted ADDITIVE keysabridged, audio_metadata: the unified search row gained the two columns it lagged (EXPLAIN plan-identical — both ride the already-scanned media_items side), and the endpoint serializes the row directly, so they emit. No key removed or renamed. - Origin writer attribution (post-hardening parity) — every web-API
write endpoint now parses an ADDITIVE optional request key
origin("android" | "ios"; anything else stays "web") and records it as the writer: progress PUT →last_sync_source, bookmark/note/highlight create+PUT →last_modified_source(andorigin_sourceon bookmark creates, now normalized through the same gate). Precedence on progress: a paired X-Device-Token still outranks origin. The web SPA sends no origin — its rows are unchanged. Request-parse widening documented per rule 2; response shapes untouched; locked by TestOriginWriterAttribution.
Changelog entry — multi-file audiobooks (2026-10-18)
GET /api/media-items/:id/chaptersgained ADDITIVE response keyslayout("" for single-file books,"folder"or"zip"for multi-file books) andtracks[](index, url, title, path, start, end, duration_seconds, size_bytes; empty for single-file books). Existing keys (media_item_id, chapters, duration_seconds, codec_profile) unchanged — no pin existed on this endpoint, so no pin widened; recorded here per rule 2's spirit.- New sibling endpoint
GET /api/media-items/:id/audio/:trackIndexstreams one track of a multi-file book (Range-capable). New surface, not a modification; gated by library visibility likeServeFile(403 on hidden libraries). media_items.audio_metadatapayload gained additive JSON keyslayoutandtracks(see schema.sql comment). Consumers parse additively; stored single-file payloads are unchanged.
2026-10-19 — A2b: metadata array/model overhaul (phase 1)
media_items.series_numberwidened INTEGER → DOUBLE PRECISION across all surfaces. JSON serializers emit the same key; values are JSON numbers, so integer-valued ordinals are byte-identical ("3") and decimal ordinals (0.5/1.5) now serialize as decimals instead of truncating. Clients reading JSON numbers are unaffected; the Android client parses either form.GET /api/media-items*/ detail responses gain ADDITIVE keys:subtitle(string),explicit(bool),authors([]string, write truth — legacyauthordisplay string still emitted, now trigger-regenerated),publishers([]string; legacypublisherlikewise).POST/PUT /api/media-items*accept ADDITIVE request keyssubtitle,explicit,authors,publishers; legacyauthor/publishersingles still accepted (arrays win when both present).- Override keys unchanged:
author/publisheroverride names are preserved for existing rows and now protect the arrays.