Part 2 of the faceted plan, Phase 4 (steps 3-4). Size-only detection
was blind to metadata-sidecar edits; a size-stable book is changed by
EDITING its metadata.opf/metadata.json/cover — previously invisible
until a force rescan.
- Sidecar manifest fingerprints (media_scanner_manifest.go): the
manifest stats exactly the names the extractors consume (metadata.opf,
metadata.json, findSidecarCover patterns incl. <basename>.<ext>), so
manifest and extractors can never disagree. Shape v1:
{v, entries:[{n, s, m (unix nanos — second granularity would hide
edits landing in the backfill scan's own second), h (sha256 — covers
included: the repoint ladder needs a baseline hash for any stat-
drifted sidecar; a hashless cover turns harmless mtime drift into a
re-extraction, the exact flood this design kills)]}.
- Evaluation ladder in cost order: stat pairs identical → skip with
ZERO bytes read; stat drift → hash that sidecar (equal = host repoint
reconciliation, stat pair refreshed, no re-extraction; differing =
changed); NULL stored manifest → backfill only, NO re-extraction
(deploy-day rule: the first scan after release must not look like an
mtime flood).
- Change path: re-read full row, extractMetadata once and pass it
through updateMediaItemWithMetadata (preExtracted — metadata_overrides
stay protected), run syncPlaylistCollectionsAfterCreate (Correction 2:
a reading-order entry added to an EXISTING book now creates its
collection; add-if-missing is idempotent), store a fresh full
manifest. Creates seed their manifest at insert time.
- Multifile folder/zip books intentionally keep Phase 3 behavior: their
change path does not consume sidecar extractors, so gating them would
flip books 'changed' without picking anything up (plan correction —
the plan's §3 multifile note assumed otherwise).
- UpsertMediaScanState + GetMediaScanState queries.
Tests: media_scanner_manifest_test.go (ladder unit tests — fast path,
repoint, same-size drift, appear/vanish, unhashable legacy entries);
TestScanManifestLifecycle (seeding, unchanged-skip stability, repoint
reconciliation with stat-pair refresh assertion, same-size edit
consumed, playlist sync on the change path + idempotency, no-leak
guard). Container runner.
Part 2 of the faceted plan, Phase 3 (steps 1-2). The periodic scan
issued one GetMediaItemByFilePath per scannable file (~3,094 queries
per scan in prod, nearly all answering 'unchanged, skip') plus one
GetLibraryByFolder per file for folder resolution.
- NEW media_scan_state table: the scanner's per-item fingerprint
cache. Deliberately NOT a media_items column — the library-scoped
list emits raw sqlc rows as JSON (7537063c), so scanner bookkeeping
must never leak into API payloads nor churn the model or pins.
Manifest JSONB shape v1 documented in the schema (n/s/m/h; ev
reserved for the Phase 4+ extractor-version mechanism).
- GetScanStateByLibrary: lean bulk projection (id, file_path,
file_size, archived_at, missing_scan_count, file_sha256, manifest).
archived/hash columns ride along so the skip path's archive-restore
and hash self-heal checks stay DB-free too (refines the plan's
4-column sketch; same goal, narrower invariant).
- ScanFolders: per-run reset, per-folder library cache, one bulk load
per distinct library. processMediaFile: map hit answers existence +
size + flags with zero DB reads; force-rescan and size-change
branches re-read the full row (the change path may be expensive);
write-through on create/size-change keeps later passes honest.
- Multifile folder/zip books keep their full-row lookup: it is one
query per BOOK ROOT (not per track) and needs audio_metadata for
the duration-probe check. They join the map in Phase 4, when the
manifest compare replaces their fingerprint check.
- nil map (watcher's scanDirectory path, bulk-load failure) falls
back to per-file lookups — behavior-preserving; the failure case
logged once per scan.
Test: TestScanStateBulkLoad (unchanged rescan leaves updated_at
stable; API-created rows are seen by the bulk load; size changes
still update in place). Container runner.
John O'Keefe