processing_issues only recorded cross-library move rejections; files
that failed to import vanished into an error counter and stdout. The
ledger now records every failure class and heals itself:
Schema (in place, pre-release boundary):
- media_item_id NULLABLE + file_path TEXT (deliberately unbounded —
an over-long path is itself a failure mode; a VARCHAR(500) ledger
column died on the exact 22001 it existed to report — caught by the
new test). Row-less failures key on (library, file_path); item-backed
on (item, type). Widened dedup = COALESCE-expression unique index;
the old table UNIQUE stays (NULLs pairwise-distinct for row-less).
- Conflict-target upsert: repeat failures UPDATE the same row and
REOPEN resolved ones — never duplicates.
Wirings:
- CreateMediaItem failure (single-file + multifile createAudioBookItem)
→ row-less import_failed (error).
- Extraction hard-fail with fallback import → item-backed
metadata_extraction_failed (warning).
- Change-path re-extraction failure (the ev/sidecar guard) → item-backed
metadata_extraction_failed; LogProcessingIssue keeps its signature, a
row-less helper joins it.
Auto-resolution:
- Successful import → ResolveOpenProcessingIssuesForPath (library,
file_path).
- Successful (re)extraction → ResolveOpenProcessingIssuesForItem
(single-file change path, multifile editorial apply).
- Unchanged-skip scans do NOT resolve: extraction never re-ran, so the
degraded metadata is still degraded — the page heals exactly when the
book does.
Family lens on ALL issues surfaces (list, stats, resolve, delete, SSR
page): explicit out-of-set library id → 403, never silent. This
tightens one pre-lens behavior — a nonexistent library id returned
200-empty, now 403 (pinned, changelog entry in docs/contract_pins.md).
List is LEFT-JOINed so row-less rows appear (media_item_id "",
title falls back to the path).
Tests: full lifecycle state machine through real scans (fail → appears,
repeat → no dups, success-at-path → resolved, LEFT-JOIN list shape);
family lens 403s with a fresh non-shared admin; all 7 pre-existing
issue suites green (two expectations widened to the lens, changelogged);
adjacent suites (CrossLibraryMove, Audiobook, Calibre,
ExtractorVersion, SchemaIdempotency) green; unit suites green.
The JWT middleware validated signature + expiry and fabricated the
request user purely from token claims, so a signature-valid token for a
DELETED user kept full API access until token expiry — observed live on
hardware (a phone session survived the test suite's user-table
recreation and kept serving its true empty sets; open handoff bug 1).
An ensureUserExistsMiddleware already existed but was only wired on the
frontend SSR group (frontend.go) — every /api group used bare
createJWTMiddleware. The existence check now lives INSIDE
createJWTMiddleware's SuccessHandler so every protected group (SSR and
API alike) gets it:
- After claim validation, GetUser by the token's user_id; a missing row
clears the token cookie and answers 401 user_not_found for API calls
(the app's TokenAuthenticator then re-logins with stored credentials)
or redirects to /login?session=invalid for page requests — same
shape-aware split the error handler already uses.
- Cost: one primary-key lookup per authenticated request (~0.2ms local);
instant revocation, no TTL semantics. The redundant SSR-only
middleware is deleted (its redirect behavior is preserved inside the
guard).
The guard exposed two processing-issues tests that were unknowingly
RELYING on ghost semantics: they nested a second setupTestServer whose
bootstrap deleted the first setup's regular user, then asserted 403
from the dead token. De-nested: a createLibraryDirect helper inserts
the library under the live setup's user, no second bootstrap.
New TestDeletedUserTokenRejected pins the contract: token works while
the user exists -> 401 user_not_found after deletion. Full suite: 219
groups, zero failures.
Three issues fixed in processing_issues_test.go:
- Empty UUID: handler returns 400 (uuid.Parse rejects empty string), not 404.
Fix expectedStatus in both List and Stats validation tests.
- Path traversal: raw '../../' in URL creates extra path segments that don't
match the route. Use url.PathEscape so the string is treated as a single
path parameter, letting the handler reject it with 400.
- SQL injection: raw special characters (semicolons, quotes) caused
httptest.NewRequest to panic. url.PathEscape prevents the panic and the
handler rejects the decoded value via uuid.Parse.
- Remove unsupported 'audiobooks' library type from
TestProcessingIssuesDifferentLibraryTypes (only ebooks/comics/manga exist
in the database schema).
Replace all unhandled resp.Body.Close() calls throughout the test suite:
- Deferred calls: replace 'defer VAR.Body.Close()' with a closure that explicitly
discards the error via 'defer func(Body io.ReadCloser) { _ = Body.Close() }(VAR.Body)'
- Immediate calls: replace 'VAR.Body.Close()' with '_ = VAR.Body.Close()'
Replace all unhandled json.NewDecoder(VAR.Body).Decode(&x) calls with error capture
and require.NoError assertion. Files using httptest.ResponseRecorder (collections_preview,
processing_issues) use 'err :=' declaration; suite-style tests (scanner_integration,
dashboard_integration) use s.T() instead of t.
Added comprehensive integration tests for the new processing issues API
endpoints that track EPUB format mismatches in manga/comics libraries.
Test Coverage:
- Authentication & authorization (no auth, invalid auth, non-admin, admin)
- Input validation (malformed UUIDs, path traversal, SQL injection attempts)
- Response structure validation (fields, types, content-type)
- Cross-library isolation (ensures issues don't leak between libraries)
- All library types (ebooks, comics, manga, audiobooks)
- Edge cases and error conditions
Endpoints Tested:
- GET /api/libraries/:id/issues/list - Lists unresolved processing issues
- GET /api/libraries/:id/issues/stats - Returns error/warning/info counts
Test Implementation:
- 522 lines, 9 test functions, 30+ subtests
- Uses setupTestServer() helper for server setup
- Uses setupDeviceTest() helper for library creation
- Follows PROJECT_GUIDELINES.md requirements
- Table-driven tests with t.Run() for comprehensive coverage
- Tests all three user contexts: no user, regular user, admin
This ensures the processing issues feature is properly tested before
integration with the media scanner service.
John O'Keefe
john-okeefe