perf(scanner): bulk scan-state load — one query per library scan

Part 2 of the faceted plan, Phase 3 (steps 1-2). The periodic scan
issued one GetMediaItemByFilePath per scannable file (~3,094 queries
per scan in prod, nearly all answering 'unchanged, skip') plus one
GetLibraryByFolder per file for folder resolution.

- NEW media_scan_state table: the scanner's per-item fingerprint
  cache. Deliberately NOT a media_items column — the library-scoped
  list emits raw sqlc rows as JSON (7537063c), so scanner bookkeeping
  must never leak into API payloads nor churn the model or pins.
  Manifest JSONB shape v1 documented in the schema (n/s/m/h; ev
  reserved for the Phase 4+ extractor-version mechanism).
- GetScanStateByLibrary: lean bulk projection (id, file_path,
  file_size, archived_at, missing_scan_count, file_sha256, manifest).
  archived/hash columns ride along so the skip path's archive-restore
  and hash self-heal checks stay DB-free too (refines the plan's
  4-column sketch; same goal, narrower invariant).
- ScanFolders: per-run reset, per-folder library cache, one bulk load
  per distinct library. processMediaFile: map hit answers existence +
  size + flags with zero DB reads; force-rescan and size-change
  branches re-read the full row (the change path may be expensive);
  write-through on create/size-change keeps later passes honest.
- Multifile folder/zip books keep their full-row lookup: it is one
  query per BOOK ROOT (not per track) and needs audio_metadata for
  the duration-probe check. They join the map in Phase 4, when the
  manifest compare replaces their fingerprint check.
- nil map (watcher's scanDirectory path, bulk-load failure) falls
  back to per-file lookups — behavior-preserving; the failure case
  logged once per scan.

Test: TestScanStateBulkLoad (unchanged rescan leaves updated_at
stable; API-created rows are seen by the bulk load; size changes
still update in place). Container runner.
This commit is contained in:
John O'Keefe committed 2026-10-19 19:52:43 -04:00
1 parent bbedbfb13f
commit 693053a9f5
7 files changed
+512 -54

No files matched your search

+15
View File
@@ -1247,6 +1247,21 @@ CREATE TABLE IF NOT EXISTS media_item_formats (
CREATE INDEX IF NOT EXISTS idx_media_item_formats_media ON media_item_formats(media_item_id, format_type);
CREATE INDEX IF NOT EXISTS idx_media_item_formats_sha256 ON media_item_formats(file_sha256);
-- media_scan_state: the scanner's per-item fingerprint cache (faceted
-- plan Part 2). Deliberately NOT a media_items column: the library-
-- scoped media list emits raw sqlc rows as JSON, and scanner
-- bookkeeping must never leak into API payloads (nor churn the model
-- or the contract pins). One row per item; cascade on delete.
-- manifest shape v1:
-- {"v":1,"entries":[{"n":"metadata.opf","s":3013013,"m":1697500000,"h":"<sha256>"}, ...]}
-- n basename, s size, m mtime unix (CACHE ONLY — never a change
-- signal), h sha256 for text sidecars (omitted for cover images).
-- "ev" (extractor version) reserved for parser-fix re-extraction.
CREATE TABLE IF NOT EXISTS media_scan_state (
media_item_id UUID PRIMARY KEY REFERENCES media_items(id) ON DELETE CASCADE,
manifest JSONB
);
-- Create device_file_aliases table (track file paths per device for cross-device matching)
CREATE TABLE IF NOT EXISTS device_file_aliases (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),