feat(scanner): extractor-version (ev) mechanism — parser fixes reach old rows without a force rescan (faceted plan Phase 10)

Parser fixes (7e4c97f6, genre model v2) never reached rows ingested
under an older parser: the sidecar never changed, so every fingerprint
said 'unchanged'. The manifest JSONB now carries the reserved 'ev'
stamp — the parser version that produced the item's row.

Semantics (all pinned by tests):
- ABSENT ev decodes as 1 ('v1-or-unknown'); stamping is truthful-only:
  create paths + post-extraction stores stamp the setting's current
  value; deploy-day backfill and the extract-failure guard stay UNSET
  (a row never claims a version its data can't prove — a mid-backfill
  bump self-heals next poll); repoint reconciliation preserves the
  stored ev (stat-only refresh re-extracted nothing).
- NEW setting scan_extractor_version (seeded + registry + getter,
  int default 1, category scanner): a manifest whose ev != the setting
  re-extracts through the SAME change path (full extraction →
  updateMediaItemWithMetadata → fresh stamped manifest). Sidecar-less
  books get an entries-less manifest as their ev home (create: stamped;
  first-sighting: unset).
- Catch-up bounded: evReextractBudgetPerScan=100, atomic (watcher pass
  can overlap a poll scan), reset per ScanFolders — amortized +
  resumable across polls; no force rescan, no flood.
- Multifile books included: the ev gate rides multifileSidecarChanged's
  shared ladder; ev-driven manifestChanged drives the editorial apply
  + stamped re-store.

Settings API: the unified GET/PUT /api/system/settings now lists the
key (registry-driven, auto-surfaces in the admin UI). Test harness:
the registry is now Loaded + wired into the SystemSettingsHandler
(main.go parity) — settings writes through the API were 503s before.

Verified: unit (ev normalization, budget CAS, GetExtractorVersion
cache paths, registry clamping); integration state machine —
create-stamp ev=1 → equality skips (corruption probe persists) →
wiped scan-state backfills UNSET without re-extract → bump re-extracts
ONCE (sidecar value restored, ev=2) → stable (second corruption
persists); multifile twin green; adjacent suites (Audiobook, Calibre,
CrossLibraryMove, SchemaIdempotency) green; live app restart re-scans
1198 files with 0 changed (deploy-day rule), setting listed at
/api/system/settings.
This commit is contained in:
John O'Keefe committed 2026-10-20 18:24:08 -04:00
1 parent 43987262a6
commit 91b0c45dd4
10 files changed
+486 -10

No files matched your search

+7 -2
View File
@@ -61,6 +61,7 @@ ALTER TABLE system_settings ADD COLUMN IF NOT EXISTS category VARCHAR(40);
INSERT INTO system_settings (setting_key, setting_value, description, setting_type, min_value, max_value, requires_restart, category) VALUES
('scan_poll_interval_seconds', '60', 'How often to scan all libraries (seconds)', 'int', '1', '3600', FALSE, 'scanner'),
('auto_scan_enabled', 'true', 'Whether auto-scanning is enabled system-wide', 'bool', NULL, NULL, FALSE, 'scanner'),
('scan_extractor_version', '1', 'Parser version stamped on each book scan manifest; raising it re-extracts metadata from books scanned under an older parser (bounded batch per scan)', 'int', '1', '1000000', FALSE, 'scanner'),
('auto_playlist_collections', 'true', 'Auto-create curated playlist collections from sidecar series marked as reading orders', 'bool', NULL, NULL, FALSE, 'scanner'),
('default_timezone', 'UTC', 'System default timezone', 'string', NULL, NULL, FALSE, 'general'),
-- security / auth (live)
@@ -1258,8 +1259,12 @@ CREATE INDEX IF NOT EXISTS idx_media_item_formats_sha256 ON media_item_formats(f
-- would hide edits landing in the backfill scan's own second), h
-- sha256 of the content — present for ALL entries, covers included:
-- the repoint reconciliation needs a baseline hash for any stat-
-- drifted sidecar. "ev" (extractor version) reserved for parser-fix
-- re-extraction.
-- drifted sidecar. "ev" (extractor version, default 1): the parser
-- version that produced the item's row. ABSENT means "v1-or-unknown"
-- — backfill stamps are unset so a row never claims a version its
-- data can't prove; every post-extraction store stamps the setting's
-- current value. A stored ev != scan_extractor_version → re-extract
-- (bounded per scan, resumable across polls).
CREATE TABLE IF NOT EXISTS media_scan_state (
media_item_id UUID PRIMARY KEY REFERENCES media_items(id) ON DELETE CASCADE,
manifest JSONB