bookhoard/docs/contract_pins.md

# Wire-contract pins (hardening batches)

These are the response SHAPES (JSON key sets, not values) frozen by the
`TestContractPin_*` suite in `cmd/server/tests/contract_pins_test.go`
(batch H0). The plan they serve is `~/.opencode/plan/hardening-canonical-types.md`.

## Rules

1. **Frozen**: no existing response key may be RENAMED or REMOVED on any
   endpoint below (JSON or KOReader/Kobo adapters). Requests are frozen too —
   no parsing changes.
2. **Additive keys are allowed and expected** — but a batch must widen the
   `allowed` slice in the test AND record the addition in the changelog
   below, in the same commit.
3. KOReader / Kobo / progress surfaces are the hard freeze: the KOReader
   plugin is an external client whose handlers keep response shapes
   byte-identical (only their internals may route through shared funnels).
4. A behavior fix that would require narrowing or REMOVING a pinned key
   needs the user's explicit sign-off before the pin is touched — a pin
   freezing buggy behavior is a known-bugs ledger, not a license.

## Pinned surfaces

| Test | Surface | Shape |
|---|---|---|
| `ProgressWeb` | `GET /api/media-items/:id/progress` (percentage-only write) | flat map: `id, media_item_id, user_id, current_page, total_pages, last_read_at, percentage, character_offset, epubcfi, chapter, chapter_progress, format_group, total_characters, chapter_count, last_sync_device, last_sync_source, last_sync_timestamp` |
| | … conditional bolt-ons | `context_text` (when stored valid); `css_selector`, `anchor_href`, `char_offset`, rewritten `epubcfi` (when the anchor verify path fires) |
| | `PUT /api/media-items/:id/progress` response | `database.ReadingProgress` JSON (25 keys incl. `audio_offset_seconds`, viewport/zoom/scroll/panel/reading_mode) |
| | `GET` with no rows | exactly `{current_page, total_pages}` |
| `KOReaderProgressPush` | `POST /api/sync/koreader/progress` (immediate, 202) | `sync_status, books_synced, book_results[]?{sha256, book_uuid, synced}, conflicts[]?{book_uuid, conflict_type, device_progress, server_progress, resolution}, timestamp, device_updated` |
| `KOReaderProgressPull` | `GET /api/sync/koreader/metadata/:uuid` | `uuid, sha256?, title, authors, progress{percentage + optional character/epubcfi/koreader_xpointer/chapter/chapter_progress/page/total_pages}, annotations{highlights?, notes?, bookmarks?, deleted_highlights?, deleted_bookmarks?}, last_sync` |
| `KOReaderLibrary` | `GET /api/sync/koreader/library` | `library_sync[]{uuid, sha256?, title, author, content_type, percent_read, pages_remaining?, bookmark_count, last_modified}, total_books, last_sync` |
| `KOReaderBookmarksPush` | `POST /api/sync/koreader/bookmarks` | `sync_status, bookmarks_synced, notes_synced, highlights_synced, total_synced, timestamp` |
| `KoboInitialization` | `GET /api/sync/kobo/:token/v1/initialization` | `library_sync[]{ContentId, ContentType, Title, Author, PercentRead, PagesRemaining?, BookmarkCount, LastModified, EntitlementId, Revision, MimeType, FileSize, Categories?, BookhoardUUID?}, total_books, last_sync` |
| `KoboMarkupSync` | `POST /api/sync/kobo/:token/markup` | `Status, MarkupsSynced, BookmarksSynced, DeletedAnnotations?` |
| `KoboSyncFromServer` | `POST /api/sync/kobo/:token/sync-from-server` | `BooksSynced, BookmarksSent, HighlightsSent` |
| `CollectionsJSON` | `GET /api/collections/:id` (all 4 branches) | envelope `id, user_id, name, description, color, icon, auto_assign_rules, view_settings, books[], book_count, is_smart, created_at`; books = `BookInfo` |
| | `POST /api/collections/preview` | bare `{items: BookInfo[]}` |
| `SeriesJSON` | `GET /api/series/books?name=&library_id=` | `{name, library_id, books: BookInfo[], total}` |
| `SearchJSON` | `GET /api/media-items/search?q=&library_id=` | top-level array of `SearchMediaItemsUnifiedRow` (47 keys + viewer_percentage) |
| `DashboardSections` | `GET /api/dashboard/sections` | `{sections[]: {id, collection_id, is_system, title, description, icon, items: BookInfo[], view_all_url, priority}}` |
| `MediaListBothBranches` | `GET /api/media-items` (unscoped) | `{data: [38-key resolved map]}` (intentionally divergent from the scoped branch until the row-shape collapse batch) |
| | `GET /api/media-items?library_id=&sort=` | `{data: [raw ListMediaItemsSortedRow, 70 sqlc keys]}` |
| `SSRPages` | `/bookshelf`, `/collections`, `/collections/:id`, `/series`, `/series/detail`, `/tags/detail` | 200 + stable title/substring markers |

`BookInfo` (every card surface): `media_item_id, title, author,
cover_image_path, has_conflict, format_group, badge, viewer_percentage,
library_name`.

## Changelog (additive keys per batch)

- **H0 (initial pin)** — the shapes above, captured on the unmodified tree
  at commit `b4ae423`.
- **H1 (progress canonicalization)** — `GET /api/media-items/:id/progress`
  Whitelisted ADDITIVE keys (fields the `GetUniversalProgressRow` already
  carries, now surfaced by the shared row serializer):
  `audio_offset_seconds, reading_mode, viewport_x, viewport_y, zoom_level,
  scroll_position_x, scroll_position_y, panel_number`. No key removed or
  renamed; the no-rows and PUT-response shapes are untouched. The web PUT
  binds `SaveProgressRequest` directly — its parse set for existing
  clients is IDENTICAL to the old DTO (verified against web/src
  reader.ts:3113-3143 and audio.ts:518-520); keys the web never sent
  (viewport_x/y, panel_number) merely become honor-able. All other pins
  unchanged.
- **H3-prep (web annotation CRUD freeze)** — NEW pinned surfaces, captured
  pre-refactor per the pin-first doctrine: `POST/PUT/DELETE
  /api/media-items/:id/{notes,highlights,bookmarks}` — create 201 / update
  200 / delete 204, response bodies are the full sqlc row models
  (MediaNotes 19 keys, MediaBookmarks 20 keys, MediaHighlights 27 keys —
  listed in contract_pins_test.go). No keys removed; nothing was pinned
  here before. Capturing these immediately caught a LIVE server bug:
  `applyNoteLWW` omitted `UserID` from its
  `UpdateMediaNoteForSync` params, so every dedup-hit note re-save
  (same content+position ⇒ same key) returned 500
  ("update note: no rows in result set") — the LWW update could never
  land. Fixed in the same commit (server-side fix, freely allowed);
  the pins are its regression net.
- **H3 (PUT fixes routed through the sync service)** — `PUT
  /api/media-items/:id/bookmarks/:bookmarkId` now parses ADDITIONAL
  optional request keys mirroring its create sibling:
  `cfi_position, page_number, chapter_number, percentage,
  chapter_reference` (all zero-value-preserving: omitted keys coalesce
  with the stored row inside the service, so legacy clients that send
  only title/notes/position keep every stored locator — verified by
  TestH3_BookmarkPUTPreservesLocators). The PUT lands as an id-targeted
  LWW edit under the row's stored dedup key (BookmarkID/NoteID mirror
  SaveHighlightRequest.HighlightID), so `last_modified_source` /
  `last_modified_at` are now maintained. Note PUT unchanged on the wire
  (content+position) but routed through the service the same way.
  Response shapes unchanged (same row models — pins green).
- **H4 (unified row completion)** — `/api/media-items/search` rows[]
  whitelisted ADDITIVE keys `abridged, audio_metadata`: the unified
  search row gained the two columns it lagged (EXPLAIN plan-identical —
  both ride the already-scanned media_items side), and the endpoint
  serializes the row directly, so they emit. No key removed or renamed.
- **Origin writer attribution (post-hardening parity)** — every web-API
  write endpoint now parses an ADDITIVE optional request key `origin`
  ("android" | "ios"; anything else stays "web") and records it as the
  writer: progress PUT → `last_sync_source`, bookmark/note/highlight
  create+PUT → `last_modified_source` (and `origin_source` on bookmark
  creates, now normalized through the same gate). Precedence on progress:
  a paired X-Device-Token still outranks origin. The web SPA sends no
  origin — its rows are unchanged. Request-parse widening documented per
  rule 2; response shapes untouched; locked by
  TestOriginWriterAttribution.

## Changelog entry — multi-file audiobooks (2026-10-18)

- `GET /api/media-items/:id/chapters` gained ADDITIVE response keys
  `layout` ("" for single-file books, `"folder"` or `"zip"` for
  multi-file books) and `tracks[]` (`index, url, title, path, start,
  end, duration_seconds, size_bytes`; empty for single-file books).
  Existing keys (`media_item_id, chapters, duration_seconds,
  codec_profile`) unchanged — no pin existed on this endpoint, so no
  pin widened; recorded here per rule 2's spirit.
- New sibling endpoint `GET /api/media-items/:id/audio/:trackIndex`
  streams one track of a multi-file book (Range-capable). New surface,
  not a modification; gated by library visibility like `ServeFile`
  (403 on hidden libraries).
- `media_items.audio_metadata` payload gained additive JSON keys
  `layout` and `tracks` (see schema.sql comment). Consumers parse
  additively; stored single-file payloads are unchanged.

## 2026-10-19 — A2b: metadata array/model overhaul (phase 1)

- `media_items.series_number` widened INTEGER → DOUBLE PRECISION across all
  surfaces. JSON serializers emit the same key; values are JSON numbers, so
  integer-valued ordinals are byte-identical ("3") and decimal ordinals
  (0.5/1.5) now serialize as decimals instead of truncating. Clients reading
  JSON numbers are unaffected; the Android client parses either form.
- `GET /api/media-items*` / detail responses gain ADDITIVE keys:
  `subtitle` (string), `explicit` (bool), `authors` ([]string, write truth —
  legacy `author` display string still emitted, now trigger-regenerated),
  `publishers` ([]string; legacy `publisher` likewise).
- `POST/PUT /api/media-items*` accept ADDITIVE request keys `subtitle`,
  `explicit`, `authors`, `publishers`; legacy `author`/`publisher` singles
  still accepted (arrays win when both present).
- Override keys unchanged: `author`/`publisher` override names are preserved
  for existing rows and now protect the arrays.

## 2026-10-19 — A2c: collections/playlists phase

- `GET /api/collections/:id` gained ADDITIVE `up_next` (array of
  `{id, title, author, cover, format, progress}` — the static collection's
  reading queue head in position order, viewer-progress-aware; empty array
  for smart collections).
- `GET /api/dashboard/sections` rows gained ADDITIVE `is_playlist` (bool —
  collection was imported from a sidecar ordering construct) and
  `complete` (bool — hide_finished filtered every item out of a non-empty
  collection; render "playlist complete").
- `PUT /api/collections/:id/items/order` is a NEW endpoint (owner-only,
  full ordered id list → positions 1..n).
- User collections are dashboard-eligible by default (the
  show_on_dashboard gate is gone from GetUserCollectionsForDashboard);
  hiding stays dashboard-prefs (`hidden_collections`) territory.
- `GET /api/media-items` (unscoped resolved-map branch) and
  `GET /api/media-items/:id` gained the same ADDITIVE keys as the raw
  branch: `subtitle`, `explicit`, `authors`, `publishers`.
- `GET /api/media-items/search` rows gained the same ADDITIVE keys plus
  the search pairs `authors_search`, `publishers_search`,
  `narrators_search` (the unified SELECT now carries the new columns).
- `author_filter` / `publisher_filter` are now per-element against the
  arrays; a comma-joined value is OR-within-facet multi-select. The
  param NAMES and single-value behavior are unchanged.

## 2026-10-19 — Faceted navigation Batch A: new filter facets

- `GET /api/media-items/search` accepts ADDITIVE REQUEST params (query
  string, response shapes unchanged): `story_arc_filter`,
  `imprint_filter`, `edition_filter` (scalar-fuzzy),
  `age_rating_filter`, `format_group_filter`, `manga_type_filter`,
  `reading_direction_filter` (exact), `explicit_filter`,
  `abridged_filter` (bool: absent = unset, "true"/"false" = bound).
  A comma-joined value is OR-within-facet multi-select on every new
  TEXT facet (the SQL splits via string_to_array). `services/facet_registry.go`
  is the Go-side source of truth; the drift-guard unit test
  (`TestFacetRegistrySQLDrift`) pins registry ↔ SQL agreement.
- Genre model v2 changes SCANNER OUTPUT (not a wire contract): sidecar
  genre selection now stores only top-level shelves in `genre`
  (subgenres become tags mapped to their parent shelf; unknown-only
  input leaves `genre` empty). Existing rows are unaffected until a
  rescan/re-extract.

## 2026-10-19 — Faceted navigation Batch B/C: chips, FacetLink, facets endpoint

- `GET /api/media-items/facets?facet=X&library_id=...` is a NEW pinned
  surface (registry-driven value+count listing for the filter panel):
  200 → `{facet, values[]: {value, display?, count}}` (display present
  only for language codes → names); 400 for unknown facets and for
  bool/range facets (they render as toggles, not lists); 403 for an
  explicit library_id outside the requester's visible set.
- `GET /api/media-items/search` (JSON) and its HTMX grid partial now
  share ONE param parser (`handlers.ParseMediaSearchParams`). Behavior
  notes: the HTMX branch previously dropped `publisher_filter` and
  `narrator_filter` silently — it now honors every facet param; and an
  explicit hidden-library `library_id` is now a 403 on BOTH branches
  (was: silent empty set on JSON). Response shapes are untouched; the
  search row pin (47 keys) is unchanged.
- `language_filter` now normalizes comma elements through the ingest
  language map, so display names ("English") match rows stored as ISO
  codes ("en"). The param name is unchanged; single-code values behave
  identically.
- Web: the bookshelf gains an active-filter chips bar (client-local
  form state rendered by Alpine; templ owns the markup) and
  `applyURLFilters` accepts the new facet params.

## 2026-10-19 — Scanner Phase 4: manifest-gated sidecar detection

- No wire change: `media_scan_state` lives in its own table, so the
  raw-row list JSON cannot contain scanner state (guarded by an
  assertion in TestScanManifestLifecycle).
- Scanner BEHAVIOR: size-stable books are now checked against a stored
  sidecar fingerprint (`{"v":1,"entries":[{n,s,m-nanos,h-sha256},...]}`).
  Stat-pair-identical → skip with zero reads; stat drift → hash
  reconciliation (equal hash = host repoint, stat pair refreshed, no
  re-extraction; differing hash = metadata re-extract + fresh manifest,
  metadata_overrides protected, playlist collections synced on the
  change path — a "(Reading Order)" entry added to an EXISTING book now
  creates its collection). NULL stored manifest → backfill only, no
  re-extraction (deploy-day rule). Multifile folder/zip books keep the
  Phase 3 behavior (their change path does not consume sidecar
  extractors yet — gating them would flip books "changed" for nothing).

## 2026-10-19 — Scanner Phase 5: quiet scans, coalescing, watcher probe

- `POST /api/scanner/scan` now answers **409** (with `active_job_id`)
  when a scan for the same library is already pending/running —
  coalescing instead of stacking a second full scan. The endpoint was
  never pinned; no existing response shape changed. Poll ticks and
  watcher events coalesce silently (no second job enqueued).
- Scan jobs carry a `trigger` label (poll/watcher/admin/initial); the
  per-scan log is now ONE summary line:
  `Scan of <folders> (trigger: <t>): N files, N changed, N new, Xs, N errors`.
  Per-file "processing/success/skipping" chatter is gone; inserts,
  changes, and errors still log individually.
- `JobResult` gains additive `job_type` (coalescing needs it; the
  scan-progress surfaces tolerate unknown keys).
- Watcher: a failed capability probe (one `Add` on a root folder)
  disables the watcher with ONE startup line instead of per-directory
  warning spam.

## 2026-10-20 — Stale pin repair (behavior shipped in 7e586df4)

- `GET /api/media-items/search` with an explicit `library_id` outside
  the requesting user's visible set — INCLUDING a nonexistent id —
  answers **403** (`library not accessible`), not 404. The family-lens
  standing rule (AGENTS.md) mandates 403 for every out-of-set id so
  library existence is not enumerable; Phase 1 (7e586df4) shipped it
  and documented it here, but the `invalid library_id` subtest in
  search_test.go still pinned the pre-family-lens 404. Pin widened to
  the shipped 403; no server change (discovered during the Phase 6
  full-suite run; the Phase 6 diff touches no handler code).
- `TestWebSocketPingPong` DELETED: it asserted a text-level
  `{"type":"ping"}` → `{"type":"pong"}` exchange that no server code
  has ever implemented and no client (web/src, Android) has ever sent.
  It passed vacuously through its read-timeout path (deadline error
  skipped the asserts) until interleaved `scan_progress` broadcasts
  from the live server's scans started reaching the socket first.
  Keepalive is the standard protocol-level ping the hub already sends.
  Never-built behavior is not a pinnable contract.

## 2026-10-20 — Scanner Phase 7: all-facets shape + SSR facet panel

- `GET /api/media-items/facets` gains a SECOND shape: with NO `facet`
  param it answers `{"facets": {<param>: [{"value","display?","count"},
  …]}}` — every listable facet in one response, counts computed under
  the ACTIVE FILTERS (the same query params the search endpoint takes;
  each facet is counted under the others — its own filter does not
  restrict its own list). `library_id` is optional on this shape:
  absent = the All-Libraries view. Family-lens unchanged: an explicit
  out-of-set `library_id` is 403 on both shapes.
- The PINNED single-facet shape (`?facet=X&library_id=…` →
  `{facet, values[]}`) is byte-identical in semantics; its
  implementation now routes through the same UNION-ALL query with
  neutral filters (the 14 per-column SQL queries were deleted — no
  wire change).
- NEW SSR partial: `GET /media-items/facet-panel` (session-cookie
  auth, bookshelf-protected group) renders the drawer's value lists —
  templ-rendered, registry order, library-type-scoped sections, active
  values marked. An out-of-set explicit `library_id` is 403 HTML.
- The 14 per-column facet SQL queries are replaced by one
  `FacetCountsUnderFilters` UNION-ALL statement; the registry drift
  guard now reflects over both generated param structs.

## 2026-10-20 — Filter plumbing: facet-panel partial removed; field-values `library_id` optional

- `GET /media-items/facet-panel` (the SSR partial documented above,
  same day) is REMOVED — the panel UX was rejected on scale grounds
  and the filter drawer becomes a rule builder with no other mode.
  Never contract-pinned while it lived (same-day entry, same-day
  removal); `TestFacetPanelPartial` was deleted with the behavior.
  The all-facets JSON endpoint and its two shapes are UNCHANGED and
  stay pinned; `FacetCountsUnderFilters` and the registry drift guard
  keep their roles (the engine outlives the panel).
- `GET /api/media-items/search?field=value` autocomplete
  (author/genre/tags/narrator/series/language): `library_id` is now
  OPTIONAL — absent means every library VISIBLE to the caller (the
  per-user family lens applies inside the query), fixing the All-Books
  view's "library_id is required" 400. An explicit out-of-set
  `library_id` is now a 403 on this endpoint too (was: silent empty
  result for hidden libraries — the silent pass is the anti-pattern
  the standing rule forbids). The single-facet
  `/api/media-items/facets?facet=X` shape still REQUIRES `library_id`
  — that pin is unchanged.

## 2026-10-20 — Filter rule builder: the `qf` wire param + composed engines (faceted plan Phase 9)

- NEW wire param on the search endpoint family: `qf` — a JSON rule tree
  `{mode: all|any, children: [{field, op, values} | {mode, children}]}`.
  Fields/operators are registry-typed constants
  (`services/filter_registry.go` — ~55 fields incl. every book column,
  file/lifecycle facts, the three HOLDING STRUCTURES — library,
  collections (playlists included, `excluded IS NOT TRUE` membership),
  series (primary + `alternate_info` full membership) — and the whole
  user-relative class reading the VIEWER's rows). Unknown fields or
  operators, disallowed operator/field pairs, malformed values, empty
  groups, and nesting beyond 8 are a **400** (`ErrInvalidFilterTree`),
  never a silently-ignored token. Values are always bound pgx
  parameters — an injection-shaped value is just a value.
- **Precedence (pinned)**: when a request carries BOTH `qf` and flat
  facet params, **qf wins** — the flat params are ignored by the engine.
  Flat links keep working unchanged server-side (and a tree equivalent
  to a flat link returns identical rows — pinned by test).
- The composed engines (`services/filter_query.go`) run the mandatory
  WHERE head (visibility join + archived + missing-scan — family lens)
  for search, suggestions, and count alike; row shape is identical to
  `SearchMediaItemsUnifiedRow`. A rule naming a hidden library matches
  nothing; an explicit out-of-set `library_id` stays a 403 on every
  entry point.
- NEW endpoint `GET /api/media-items/filter-count?qf=…[&library_id=…]`
  — the builder's live count: a true COUNT(*) of the rules + scope
  (not the page length). Family-lens 403 applies.
- Field-values autocomplete UPGRADED in place (same route): ANY registry
  field key in its bare form (`?note_text=…`, `?collection=…`,
  `?library=…`) routes to the field-values path; with a `qf` param the
  OTHER rules bind as filters (symmetric under-filter suggestions,
  zero-match values never offered). Without a tree, the classic six
  (author/genre/tags/narrator/series/language) keep their static
  queries and response shape — pinned contract, unchanged. ID-valued
  suggestions (library/collection/added_by) add an OPTIONAL
  `display` field to result items (`value` stays the id; `omitempty`
  elsewhere).
- The Phase-7 facet-panel partial stays removed; the all-facets JSON
  endpoint and its two shapes are unchanged (they serve flat params —
  the builder's UI does not use them).

## 2026-10-20 — Scanner Phase 10: extractor version (ev) mechanism
- `media_scan_state.manifest` JSONB gains the reserved `ev` key:
  the parser version that produced the item's CURRENT row. ABSENT
  means "v1-or-unknown" — decode normalizes it to 1.
- NEW setting `scan_extractor_version` (int, default 1, min 1,
  category scanner): the unified `GET/PUT /api/system/settings` list
  now includes this key (additive; the list was not a pinned closed
  set). Writes stay settings-API-only — direct SQL leaves the registry
  cache stale (known gotcha).
- Scanner behavior: a stored manifest whose ev ≠ the setting is
  treated as CHANGED → full re-extract + the fresh manifest stamps
  the new ev. Stamping is TRUTHFUL-ONLY: create paths and
  post-extraction stores stamp; deploy-day BACKFILL and the
  extract-failure guard store UNSET (the row never claims a version
  its data can't prove — a mid-backfill bump self-heals next poll).
  Repoint reconciliation preserves the stored ev (stat-only refresh,
  nothing re-extracted).
- Catch-up is BOUNDED per scan pass (`evReextractBudgetPerScan = 100`,
  atomic, reset at each ScanFolders): resumable across polls, no
  force rescan, no flood. Budget exhaustion leaves ev stale → later
  polls resume.
- Deploy-day rule (unchanged, re-verified live): manifests without ev
  ride v1; at default setting 1 the first post-release scan re-extracts
  NOTHING (0 changed on 1198 files).

## 2026-10-20 — Scanner Phase 11: the processing-issues ledger
- `processing_issues.media_item_id` is now NULLABLE and the table gains
  `file_path` (TEXT, deliberately unbounded — an over-long path is
  itself a failure mode the ledger must survive): ROW-LESS import
  failures (no media_items row exists) key on (library, file_path);
  item-backed rows keep (item, type). The widened dedup is the
  COALESCE-expression unique index idx_processing_issues_dedup; the old
  table-level UNIQUE stays (row-less rows have NULL items, pairwise
  distinct under unique constraints).
- `GET /api/libraries/:id/issues/list` — the media_items join is now
  LEFT: row-less issues APPEAR (they were unrepresentable before).
  Response shape unchanged for item-backed rows; row-less rows carry
  `media_item_id: ""` and a `title` fallen back to the file path.
- FAMILY LENS now applies to ALL issues endpoints (list, stats, resolve,
  delete) and the SSR page — they leak media metadata. An explicit
  library id outside the requester's visible set is a **403** (this
  tightens one pre-lens behavior: a nonexistent library id used to
  return 200-empty; it is trivially out-of-set → 403 now, pinned).
- NEW scanner wirings: CreateMediaItem failure → row-less
  `import_failed` (error); extraction hard-fail with fallback import →
  item-backed `metadata_extraction_failed` (warning); change-path
  re-extraction failure → item-backed `metadata_extraction_failed`.
- Auto-resolution: a successful import resolves row-less issues by
  (library, file_path); a successful (re)extraction resolves the item's
  open issues. Unchanged-skip scans do NOT resolve (extraction never
  re-ran — the degraded metadata is still degraded). A repeat failure
  reopens a resolved row (upsert sets resolved=false).

## 2026-10-21 — Phase 13: force rescan = original scan; the manual-edits filter
- **Force rescan contract change (owner-directed)**: force rescan now
  behaves like an original import — the full metadata chain (sidecars →
  folder structure → tags → fallbacks) re-lands on MULTI-FILE books
  too (the pre-Phase-13 editorial-keep skip is retired; the owner's
  metadata_overrides count in the single digits, and overrides are the
  only protection, honored inside the apply). A force pass also STAMPS
  each re-derived book's manifest with the current parser version
  (success-only) and closes its open processing issues. Single-file
  force rescan already behaved this way — now pinned by test.
- **NEW filter field** `manually_edited` ("Manually edited", boolean):
  is_true = `cardinality(mi.metadata_overrides) > 0`, is_false = `= 0`.
  Composed-engine (qf) only — no flat param, no value menu, no
  suggestion surface; family lens inherited.
- Deployment ordering note (owner Q&A): set scan_extractor_version
  BEFORE Rescan All — force rescan stamps books with the current
  setting, so bump-first + force = one pass of work; force-first +
  bump = the trickle re-derives everything again (wasted, not harmful).

## 2026-10-27 — Search-progress batch: viewer progress on cards
- **`SearchJSON`**: `SearchMediaItemsUnifiedRow` + `viewer_percentage`
  (float 0..1, NULL when the requester never opened the book). The
  query LEFT JOINs `reading_progress` on the SAME `user_id` param the
  library-visibility join already binds — one requester param, no new
  wire inputs; the join is row-neutral (UNIQUE(media_item, user)).
- **`BookInfo`** (CollectionsJSON / SeriesJSON / preview /
  DashboardSections / search cards): + `viewer_percentage` (JSON null
  when nil) and + `library_name` (empty on feeds that don't load it —
  only the search-unified adapter populates both today). Adapters
  other than BookCardsFromUnified keep emitting null/""; templ cards
  render the progress bar/completed check only when populated.
- The header-search dropdown now renders the templ `SearchResults`
  partial (layout=dropdown on the same endpoint, HX-Request) instead
  of the retired innerHTML renderer in web/src/search.ts — same
  endpoint, same auth, content server-rendered.

## 2026-10-29 — RLS visibility: item-scoped gates may answer 404 as well as 403

With library visibility enforced by Postgres RLS (see
database/schema/schema.sql), a hidden library's rows are INVISIBLE to
the engine — item-scoped endpoints (chapters, tracks, download, detail)
can no longer run their Go-level 403 gate because the row never arrives.
Both answers are gated: 403 (Go-level explicit-library checks remain on
listing endpoints) and 404 (row-level invisibility — no existence leak).
Pins asserting 403 on item-scoped content endpoints widen to accept
{403, 404}. Listing/scoped endpoints keep the strict 403 contract.

## 2026-10-29 — Podcast episode columns join media_items (additive widening)

The podcast integration (plan podcast-integration.md tasks 1.1/1.10) adds
fifteen nullable/no-null-default columns to `media_items` — they surface in
raw sqlc rows ONLY (`library-scoped list emits raw sqlc rows`), never in the
resolved unscoped map:

`source` ('scan' default), `source_feed_id`, `episode_guid`, `feed_position`,
`season_number`, `episode_number`, `enclosure_url`,
`declared_duration_seconds`, `transcript_url`, `chapters_url`, `hosts`,
`guests`, `download_state`, `download_error`, `download_attempts`.

Scan-sourced rows carry the inert defaults ('scan' / NULLs / 'pending');
raw-row consumers that filter `source='scan'` (scanner paths) are unaffected.
Nothing removed, nothing renamed.

## 2026-10-30 — download_queue_pos joins media_items (additive widening)

§0 item 4 of podcast-integration.md: the download pool's monotonic
queue ticket. `download_queue_pos BIGINT` (NULL for scan rows and
never-queued episodes) is stamped from the `podcast_queue_seq` SEQUENCE
at enqueue time so claim order = enqueue order (the effective listening
order). Raw-row pins (TestContractPin_MediaListBothBranches →
rawRowKeys) whitelist the key. Nothing removed, nothing renamed.

## 2026-10-30 — episode collections (additive widening, plan item 2/2.3/2.4)

Two shapes widen, both ADDITIVE — nothing removed, nothing renamed:

- **Collections envelope** (`GET /api/collections/:id` → envelopeKeys)
  gains `rule_match_mode` (smart-collection match semantics, 'any'
  legacy default | 'all'). Static collections emit the default value.
- **BookInfo** (every card surface → bookInfoKeys) gains four STABLE
  keys — `is_episode`, `feed_title`, `download_state`, `hidden` — the
  episode-collections extras. Adapters that predate episode awareness
  emit their zero values (`false`/`""`), matching the
  viewer_percentage precedent (nil-representable, key always present).

## 2026-10-30 — collections envelope gains filter_tree (additive widening, item 3.2)

`GET /api/collections/:id` adds `filter_tree` to the envelope: the
unified filter tree when the smart collection carries one (item 3.2 —
tree-driven membership), `null` for legacy/static collections. The
tree is the same qf DSL as saved filters (`{scope?, mode, children}`).
Nothing removed, nothing renamed.

## 2026-10-30 — rules endpoints RETIRED (filter-system unification, item 3.4)

The per-rule CRUD endpoints and the preview endpoint are DELETED — the
unified filter tree (item 3.2) is the only smart-membership authoring
surface, and the rules PAGE is now the filter builder in the
collection's scope:

- `POST /api/collections/:id/rules`, `PUT/DELETE
  /api/collections/:id/rules/:ruleId` — gone (legacy rule arrays still
  EVALUATE until the collection is rewritten; conversions run at the
  builder's first Apply via RulesToFilterTree).
- `POST /api/collections/test-rules` — gone (the builder's live count
  via /api/media-items/filter-count replaces the preview).
- `POST /api/collections/preview` STAYS (the dashboard
  custom-section-builder still consumes it — flagged for its own
  migration, not silently broken).
- Legacy compatibility KEPT: `auto_assign_rules` still round-trips in
  the collections envelope; `PUT /api/collections/:id/match-mode`
  still serves rules-array collections.

Test-side deletions follow the asymmetry rule: subtests exercising the
deleted endpoints are gone; their surviving SEMANTICS are pinned in
their tree form (TestSmartCollections rewritten on trees,
TestEpisodeCollectionsFilterTree, TestPodcastSmartCollections).

## 2026-10-31 — collections RLS: cross-user detail/writes tighten to 404/no-op

`collections` and `collection_items` joined the RLS-governed tables
(policies `p_collections_*` / `p_collection_items_*`, predicates
`app_can_read_collection` / `app_can_write_collection` — the same
doctrine as the media_items family lens, 2026-10-29). Contract
changes:

- `GET /api/collections/:id` for another user's collection: was 200
  (by-id readability), now 404 (row invisible).
- `PUT /api/collections/:id` cross-user: was ungated success (no
  app-level ownership check), now 404 — the RLS update touches no
  rows, and the handler maps ErrNoRows to 404.
- `POST /api/collections/:id/books` cross-user: was ungated, now 404
  (rejectSmartMutation's GetCollectionData is RLS-invisible).
- `DELETE /api/collections/:id` cross-user: still 204, but now a true
  DB-level no-op (zero rows deleted) instead of an app-level free hit.
- `PUT /api/collections/:id/view-settings` cross-user: 404 (the owner
  gate stays as defense-in-depth; the DB is enforcement of record).
- `PUT .../match-mode` and `.../items/order` keep their app-level 403
  gates; the DB now backs them.
- Reads: owner or admin (policy escape); non-owner non-admin users see
  nothing, anonymous included. Future sharing rides the predicate
  functions — one EXISTS branch in app_can_read/write_collection, no
  policy migration.

Test-side: TestCollectionsRLS pins the cross-user matrix end-to-end;
fixture writes to non-system collections run as the row's OWNER
(scopeExecAs/ownerSeedCtx) because the admin write-escape covers
system collections only.

## 2026-10-31 — legacy collection rules surface RETIRED (tree-only storage)

The pre-tree rules machinery is gone — no production row ever carried a
rules array or a non-default match mode (verified in the dev DB), the
web never reads the envelope keys, and the Android app consumes none of
it. Deletions:

- `PUT /api/collections/:id/match-mode` — endpoint retired (it had no
  doc page). The tree's root `mode` owns any|all semantics now.
- Envelope: `auto_assign_rules` and `rule_match_mode` are REMOVED from
  the create/list/detail/update payloads. This entry retracts the
  2026-10-30 pins that listed them. `collection_type` and `is_smart`
  stay.
- Request: `auto_assign_rules`/`rule_match_mode` are no longer accepted
  on create/update. A create carrying `auto_assign_rules` answers 400
  naming the retirement — the custom-section builder's legacy save
  (already failing to bind, verified live) fails loudly instead of
  silently creating a hollow static section.
- Storage: the `collections.auto_assign_rules` and
  `collections.rule_match_mode` columns are DROPPED (idempotent
  `DROP COLUMN IF EXISTS` in schema.sql).
- Service: the rules CRUD layer (AddRule/SetRuleEnabled/RemoveRule +
  validateRule + the UpdateCollectionRulesJSON query),
  ParseRules/EnabledRules/IsSmartCollection, RulesToFilterTree (+ the
  SSR preload branch), and the legacy branch of EvaluateSmartMembership
  are deleted. IsCollectionSmart = tree presence.

KEPT (live consumers): `POST /api/collections/preview` still accepts
`rules` and evaluates them in memory (the custom-section builder's
preview button); `include_auto` on the list stays (the Android app
sends it). The builder's SAVE remains the flagged §12 tree migration.

## 2026-10-31 — custom-section builder save: the flagged tree migration lands

Supersedes the same-day retirement entry's loud-400 stance for the
builder's payload. `POST /api/collections` now ACCEPTS the builder's
shape and adapts it at write time:

- `auto_assign_rules` (JSON-encoded array — string or bare) converts to
  a filter tree via the restored `RulesToFilterTree`; converted trees
  validate under `ParseFilterTree`, so an unhonorable rule is a 400 at
  write naming the problem.
- `match_type` seeds the tree's root mode (`any` default).
- `manual_book_ids` attach to the created collection after create
  (unparsable ids skipped — the AddBooks leniency).
- `show_on_dashboard` persists (new create field).
- `library_id` remains selection context only (not stored).

Dashboard sections: smart user sections evaluate their tree LIVE —
`GetUserCollectionItems` merges stored membership with the composed
engine's matches (deduped, library-scoped, capped at the section
limit). This exposed and fixed a real wiring bug: every production
`NewDashboardService` call omitted the search service, so tree
evaluation through the dashboard could never work (the test harness
wired it correctly — the gap was production-only). The dashboard
handler and both inline sites now receive it, and evaluation failures
log instead of silently emptying a section.