John O'Keefe
91b0c45dd4
Parser fixes (7e4c97f6, genre model v2) never reached rows ingested
under an older parser: the sidecar never changed, so every fingerprint
said 'unchanged'. The manifest JSONB now carries the reserved 'ev'
stamp — the parser version that produced the item's row.
Semantics (all pinned by tests):
- ABSENT ev decodes as 1 ('v1-or-unknown'); stamping is truthful-only:
create paths + post-extraction stores stamp the setting's current
value; deploy-day backfill and the extract-failure guard stay UNSET
(a row never claims a version its data can't prove — a mid-backfill
bump self-heals next poll); repoint reconciliation preserves the
stored ev (stat-only refresh re-extracted nothing).
- NEW setting scan_extractor_version (seeded + registry + getter,
int default 1, category scanner): a manifest whose ev != the setting
re-extracts through the SAME change path (full extraction →
updateMediaItemWithMetadata → fresh stamped manifest). Sidecar-less
books get an entries-less manifest as their ev home (create: stamped;
first-sighting: unset).
- Catch-up bounded: evReextractBudgetPerScan=100, atomic (watcher pass
can overlap a poll scan), reset per ScanFolders — amortized +
resumable across polls; no force rescan, no flood.
- Multifile books included: the ev gate rides multifileSidecarChanged's
shared ladder; ev-driven manifestChanged drives the editorial apply
+ stamped re-store.
Settings API: the unified GET/PUT /api/system/settings now lists the
key (registry-driven, auto-surfaces in the admin UI). Test harness:
the registry is now Loaded + wired into the SystemSettingsHandler
(main.go parity) — settings writes through the API were 503s before.
Verified: unit (ev normalization, budget CAS, GetExtractorVersion
cache paths, registry clamping); integration state machine —
create-stamp ev=1 → equality skips (corruption probe persists) →
wiped scan-state backfills UNSET without re-extract → bump re-extracts
ONCE (sidecar value restored, ev=2) → stable (second corruption
persists); multifile twin green; adjacent suites (Audiobook, Calibre,
CrossLibraryMove, SchemaIdempotency) green; live app restart re-scans
1198 files with 0 changed (deploy-day rule), setting listed at
/api/system/settings.
69 lines
2.0 KiB
Go
69 lines
2.0 KiB
Go
package services
|
|
|
|
// Unit tests for the extractor-version (ev) mechanism (faceted plan
|
|
// Phase 10): the wire normalization (absent = v1-or-unknown = 1) and the
|
|
// per-scan catch-up budget. The full scan-state machine (ev mismatch →
|
|
// re-extract once → stable) lives in cmd/server/tests
|
|
// (TestExtractorVersionReextract) because the store paths need a real DB.
|
|
|
|
import (
|
|
"testing"
|
|
)
|
|
|
|
func TestManifestEvNormalization(t *testing.T) {
|
|
if got := manifestEv(nil); got != 1 {
|
|
t.Errorf("nil manifest rides v1: got %d, want 1", got)
|
|
}
|
|
cases := []struct {
|
|
raw string
|
|
want int
|
|
}{
|
|
{`{"v":1}`, 1}, // old shape: ev absent → v1-or-unknown
|
|
{`{"v":1,"ev":0}`, 1}, // explicitly unset → same
|
|
{`{"v":1,"ev":1}`, 1}, // baseline parser stamp
|
|
{`{"v":1,"ev":2}`, 2}, // post-bump stamp
|
|
{`{"v":1,"entries":[],"ev":3}`, 3}, // sidecar-less ev home
|
|
}
|
|
for _, tc := range cases {
|
|
m := decodeSidecarManifest([]byte(tc.raw))
|
|
if m == nil {
|
|
t.Fatalf("%s must decode", tc.raw)
|
|
}
|
|
if got := manifestEv(m); got != tc.want {
|
|
t.Errorf("manifestEv(%s) = %d, want %d", tc.raw, got, tc.want)
|
|
}
|
|
}
|
|
}
|
|
|
|
func TestEvBudgetTake(t *testing.T) {
|
|
scanner := &MediaScanner{}
|
|
scanner.evBudgetRemaining.Store(2)
|
|
if !scanner.evBudgetTake() {
|
|
t.Error("budget 2: first take must pass")
|
|
}
|
|
if !scanner.evBudgetTake() {
|
|
t.Error("budget 1: second take must pass")
|
|
}
|
|
if scanner.evBudgetTake() {
|
|
t.Error("budget 0: third take must fail")
|
|
}
|
|
if got := scanner.evBudgetRemaining.Load(); got != 0 {
|
|
t.Errorf("budget must land at exactly 0, got %d", got)
|
|
}
|
|
}
|
|
|
|
func TestEvBudgetTake_FailsFastAtZero(t *testing.T) {
|
|
scanner := &MediaScanner{}
|
|
scanner.evBudgetRemaining.Store(0)
|
|
// Repeated takes at 0 must fail WITHOUT driving the counter negative
|
|
// (the CAS loop only fires when cur > 0).
|
|
for i := 0; i < 5; i++ {
|
|
if scanner.evBudgetTake() {
|
|
t.Fatalf("take %d must fail at budget 0", i+1)
|
|
}
|
|
}
|
|
if got := scanner.evBudgetRemaining.Load(); got != 0 {
|
|
t.Errorf("budget must stay 0, got %d", got)
|
|
}
|
|
}
|