The dashboard section builder — feeding both the dashboard JSON pin
surface and the SSR dashboard — now calls BookCardsFromMediaItems on
ds.Items. libTypes already arrives as a BuildSections parameter
(once per request); the literal only duplicated the badge lookup.
GET /api/series/detail?name=… books array via BookCardsFromMediaItems.
Series had the third LibraryTypeMap call INSIDE its per-item loop —
ListLibraries ran once per book on the page; now once per request.
POST preview-feed had the ladder's 14th hand-rolled literal. Now
BookCardsFromMediaItems + one LibraryTypeMap per request; the response
carries format_group/badge values for the first time (all seven keys
were already emitted as zero-value JSON, so this is a value change on
a pin-frozen shape, not a key change).
GET /api/collections/:id books — all four branches now build cards via
the cards.go adapters. LibraryTypeMap hoisted to once per handler (it
was 1 query per item in the smart and no-library branches). The system
branch — which before stamped cards BARE (no format_group, no badge —
the ladder's only JSON gap) — now emits the same card shape as its
siblings by feeding section.Items through BookCardsFromMediaItems,
matching what BuildSections produces for the identical data. The
collection-error seed dies via the adapters' non-nil empty slices;
the no-matching-section nil result is preserved as-is (that branch
never seeded — today's 'items': null stays today's 'items': null).
Pins: CollectionsJSON, DashboardSections, MediaListBothBranches green.
BookInfo, LibraryTypeMap, and MediaBadgeKind move verbatim out of
collections.go into handlers/cards.go (the card package surface, same
package: no import-graph or templ changes), joined by the H2 builder:
- buildBookCard (unexported): the ONLY place BookInfo is constructed,
fed by cardSource; cover URL via ResolveMediaURL, badge via
MediaBadgeKind — both inside the builder, not per call site.
- Four exported adapters normalize the four real feed shapes:
database.MediaItems (sections/smart/series/tags/preview),
GetCollectionItemsForDashboardRow (collection dashboard branch),
GetCollectionItemsRow (collection-items joins — MediaItemID field),
SearchMediaItemsUnifiedRow (badge rides its LibraryTypeName column,
so no LibraryTypeMap query at all).
- Adapters return non-nil empty slices, which retires the per-site
[]BookInfo{} empty-seed assignments when call sites migrate.
No call site changes in this commit — the 20 literals and 5 seeds are
migrated one surface per commit next, each with the H0 pin suite.
Verified: build, vet (only the two pre-existing flagged saveCoverImage
warnings), and 14/14 contract pins — the key freeze is byte-identical.
Found while verifying H1's "app offline-play -> flush -> position lands"
with the emulator path: the flush has been dead code in two independent
ways, invisible because the first break made the second unreachable.
1. ListPendingSyncQueueItems matched `device_id = $1` with a NULL $1
(the processor passes DeviceID{Valid:false} to enumerate all devices;
uuid = NULL matches no rows). The ticker therefore processed nothing,
ever. The WHERE now treats NULL as "every device" (the same
narg::uuid IS NULL idiom the visibility queries use; EXPLAIN'd — seq
scan on a bounded table, idx_sync_queue_status_priority_created
fits). The processor also dropped its fake-pagination loop, which
re-fetched the same first batch forever once the backlog hit the
batch size (no OFFSET in the query).
2. dbItemToSyncQueueItem dropped SyncData — even with a working
enumerator, every flushed item arrived with an empty payload and
failed JSON unmarshal. Dead code harboring silent breakage, exactly
the plan's thesis.
Callers updated for the renamed RowLimit param (queue handler, offline
detector). TestH1_SyncQueueFlush_LandsAndPreservesAudio drives the real
flow: web PUT seeds audio 900s, a pending checkpoint-style queue row
applies percentage 0.8 through SaveProgress, audio survives, queue row
completes.
Also fixes dashboard_test.go's stale BuildSections signature (the
package's test binary has been unbuildable) — user-approved: tests
designed against old shapes are not worth keeping.
Batch H1 step 4:
- Handler GetUniversalProgress (progress.go) deleted — no route mounts
it; only /api/progress/:id/history (GetProgressHistory) and the SSR
/progress page exist. Note: the bruno universal-progress suite
documents GET/PUT /api/progress/:id + /api/progress/device/:device_id
— those routes do NOT exist server-side; that suite is stale
documentation, flagged in the plan.
- Handler GetAllProgress deleted — unregistered near-copy of
GetAllProgressData (which the /progress SSR page uses and stays).
- series.go: dropped the `_ = libTypes` no-op (libTypes IS used by the
Badge line right below — the assignment stays).
- router/helpers.go: visibleLibraryIDsFor's signature and first
statement were fused onto one line; split for readability. No
behavior change.
Batch H1 step 3b — THE data fix. Conflict resolution was the one live
erasure writer: applyProgressResolution and its verbatim twin
applyResolution both wrote the winner via a raw UpdateUniversalProgress
whose param list omits audio_offset_seconds/context_text/viewports, and
the SQL's DO UPDATE SET ... = EXCLUDED... NULLed exactly those. The web
UI resolves through POST /api/conflicts/:id/resolve and
/api/conflicts/bulk-resolve (main.js resolveConflict/bulkResolve), and
the reader's mark-as-read fires a resolution first when the reader SSR
stamps data-conflict-id/-winner — so every resolution today erased the
audiobook position and reader context.
Now: ProgressService is a required ConflictHandler dependency; BOTH
call sites (single resolve + bulk resolve) share one
applyProgressResolution that funnels through SaveProgress (the verbatim
twin is deleted). Deliberate choices, per the plan:
- SkipConflictDetection on the resolution write: both flows apply
BEFORE the conflict row is marked resolved, so the funnel's LWW
detector would otherwise file a duplicate pending conflict
(HasRecentConflictResolution is false at apply time).
- Source "conflict_resolution" — the row's last-write attribution; the
winner stays durably recorded in resolution_data.winner.
- Broadcast: true — live devices jump to the winning position.
- The winner snapshot maps percentage/epubcfi/context_text/chapter/
character/page/total_pages; a winner without a CFI clears the locator
pair (ClearLocator) instead of re-pairing with the loser's stale
anchor. Everything the snapshot doesn't carry SURVIVES via the LWW
seed: audio_offset_seconds above all. chapter_progress keeps its
stored value (was overwritten with the book percentage), an absent
percentage survives (was written as 0%).
- Accepted: the resolution captures one reading_history session
(attributed to conflict_resolution; the 2h idle ceiling applies).
Verification: TestH1_ConflictResolutionPreservesAudio drives the real
flow — web push with audio 300s, diverging push from a second source
detects a conflict, resolve with the second source as winner, GET shows
percentage 0.3 AND audio_offset_seconds 300 (previously NULLed), and no
duplicate pending conflict remains. Full contract pin suite green.
The host-run TestConflict*/TestProgress* failures are environmental
(container-only /app/uploads path + db hostname; fail identically at
baseline b4ae423) — the same suites pass in the test-runner container.
Batch H1 step 3a. The ceiling census (exhaustive UpdateUniversalProgress
grep) found 9 nil-guarded `if progressSvc != nil { funnel } else { raw
erase-write }` fallbacks — the survey's "four" under-counted kobo.go,
which has FOUR sites (Markup ReadingSync, Markup last-read-place,
AnalyticsGettests, SyncFromServer), plus one each in media.go, queue.go
and koreader.go. All unreachable: the service is created unconditionally
at main.go:127 and wired into every handler and the queue processor; the
test harness wires them too.
The erasure pattern is now unrepresentable:
- NewMediaHandler / NewKOReaderHandler / NewKoboHandler / both
NewSyncQueueProcessor flavors take the ProgressService as a REQUIRED
constructor arg; the SetProgressService setters are gone.
- Every fallback body (raw UpdateUniversalProgress with explicit-NULL
pgtypes for viewport/scroll/panel/reading_mode/zoom and no
audio_offset_seconds/context_text) is deleted; each write funnels
through SaveProgress with the LWW seed, so fields the writer doesn't
know survive.
- Removed the now-dead koreader pg-type helpers (textPtrToPgText,
intPtrToPgInt4, int64PtrToPgInt8) and their self-tests.
Behavior deltas (stored-row content, no wire shape changes —
KOReader/Kobo response pins stayed green):
- KOReader fallback used to write explicit NULL viewports and drop the
context_text the koplugin sends; the funneled mainline already maps
book.ContextText.
- Queue flushes no longer overwrite chapter_progress with the book
percentage (bogus data; preserved via the seed).
- A web PUT without percentage no longer writes 0% via the deleted
fallback; percentage seeds from the stored row.
Contract suite: KOReader×4 + Kobo×3 pins green.
Batch H1 steps 1-2 of the hardening plan. Wire contracts frozen; the
pinned shapes are unchanged modulo the whitelisted additive GET keys
(docs/contract_pins.md).
1. One canonical request struct. The web PUT binds SaveProgressRequest
directly (identity/attribution fields are json:"-" so they stay
server-owned) and copies its server fields once — the hand-rolled DTO
and the 20-line field-copy block are gone. Parse set for existing
clients is IDENTICAL to the old DTO (verified in web/src: the ebook
reader sends percentage/current_page/total_pages/reading_mode/epubcfi/
context_text/chapter/zoom_level — reader.ts:3113-3143; the audiobook
reader sends percentage/chapter/audio_offset_seconds/elapsed_seconds/
reading_mode — audio.ts:518-520).
2. GET /api/media-items/:id/progress now serializes through one
row-struct-driven builder instead of a hand-map, so a row column
cannot be silently forgotten again (the audio_offset_seconds
omission). Existing keys are verbatim; the row already carried
audio_offset_seconds/reading_mode/viewport_x-y/zoom_level/
scroll_position_x-y/panel_number and they surface as WHITELISTED
additive keys. The web audiobook reader already reads
audio_offset_seconds (audio.ts:219) — with the key absent,
durationless books resumed at chapter start (offset always 0); they
now resume correctly. The conditional context_text bolt-on and the
server-derived css_selector/anchor_href block are unchanged —
restore precedence stays percentage-first, never re-derived.
The /stats page gains a second tab (Reading | Library) implementing the
Audiobookshelf-style library dashboard (HANDOFF item 9): media_items
INVENTORY aggregates — items, hours/pages, distinct authors, size,
distinct narrators, Top Genres / Top Authors bars, Longest / Largest
items — for one library or the All-libraries aggregate.
- GET /api/stats/library-usage?library_id=<uuid|omit>; family lens
enforced: a library_id outside the requester's visible set is a 403,
never a silent pass (AGENTS.md standing rule; 431fec0 history)
- six sqlc queries with the visibility pattern + healthy-item filter
(archived_at IS NULL AND missing_scan_count = 0); hours_mode lets the
client swap the Hours/Pages card and the Longest ranking per scope
- Reading tab unit audit (HANDOFF item 3, settled 2026-10-16): minutes
are the merged currency everywhere honest; only the PAGES unit splits
— Pages card goes mode-aware (Pages · text / Listening Time in an
audio-only pool) and the charts row gains Listening per Day (hrs)
when audio minutes exist in the range; summary daily buckets gain
audio_minutes split by the item's library type
- tab state in the URL (?tab=library&library_id=…); selector fed by the
existing /api/libraries/visibility endpoint; main.js?v=4
- integration tests: family-lens 403 pin, hidden-library exclusion
(differential), seeded text+audio aggregate correctness, auth gate
- docs/developer/api/stats/get_library_usage.md
Known-remaining suite failures are environmental: TestConflictList_Empty
+ TestWebSocketPingPong collide with the concurrently-running app session
(live emulator pushes + scans on the shared dev DB), and
TestAdminCreateUser's timezone expectation drifted from live system
settings — all three fail on the clean tree too.
A2 made narrators/narrators_search NOT NULL DEFAULT '{}' (86f0af6), but an
explicit NULL in an INSERT/UPDATE column list bypasses the column default
— the API media-create path (POST /api/media-items without narrators)
500'd on every call, taking the whole integration suite down with it
(ISBN tests, capture seeds, all direct-DB seeds).
- set_library_type_name() trigger now coalesces NULL narrators/
narrators_search to '{}' at the row level, and its update trigger
widens from BEFORE UPDATE OF library_id to BEFORE UPDATE so every
write path is covered
- defense in depth at the callers: handler create/update normalize nil
to empty slices (normalizeNarrators), scanner seeds the same
(nilToEmpty) for files that carry no narrator credits
- analytics device-usage tests move to the dedicated per-test user
(the shared admin's device list accrues live rows mid-suite — the
documented testuser flapping class)
Reader fullscreen: the topbar gains a fullscreen toggle (icon state + f
shortcut + help row; f is guarded so Ctrl/Cmd+F stays find-in-page).
Enter requests the Fullscreen API with fallbacks (desktop/Android get a
true full display; iOS Safari has no page-fullscreen API and falls back
to the soft visual mode only). Margins: the foliate paginator's closed
shadow root only yields through its observed 'margin' attribute, so
fullscreen fits a comfortable ratio (6% of the short side, floored at
48px) and floors it by the measured env insets — Android reports true
cutout insets in fullscreen, so portrait phones keep text clear of the
camera band; iOS letterboxes are already env-padded chrome. Fixed-layout
renderers skip it (fxl/webtoon pack pages edge-to-edge by design).
Exits the user makes outside the toggle (Esc, browser UI) sync via
fullscreenchange(+webkit) and restore the ordinary 48px margin; a
position-keeping re-render follows the ResizeObserver re-entry path.
Verified on-device: enter -> true fullscreen + icon swap + camera band
clear, exit -> chrome back, margins restored, position kept.
Volume: revert the stacked panel — the popup returns to the liked shape
(mute button left, horizontal dragger right, dialogue above the strip);
one-line flex restore of the pre-89665fb layout.
Manga badges: MediaBadgeKind makes the library type authoritative for
the comic/manga split — manga volumes ship as fixed_layout scans as
often as comic_archive (and manga_type is unreliable across sources:
unknown/NULL on real rows), so the old comic_archive+manga mapping left
the scan volumes unbadged. Anything in a manga library reads manga;
anything in a comics library reads comic. Verified: /dashboard lands 24
title="Manga" cover badges for the Manga library and the sections API
reports badge=manga for a fixed_layout item.
Ghost sidebar: the >=1024px media block applied body
padding-left:var(--sidebar-w) unconditionally, so pages that never
render the sidebar (logged-out pages, the audio player, the reader
itself) carried a phantom 16rem left margin that pushed content off
center on desktop widths. The body:has(.app-sidebar) rule below was
already the correct conditional source — it is now the only one.
Verified in the built stylesheet: exactly one such rule remains.
Infra: main.js ?v=3 stamp bump (reader.ts changed; hashed chunks churn
their URLs themselves).
Annotations banner: audio/text bookmarks join notes and highlights —
third row kind (bookmark glyph indicator, H:MM:SS / Page N / Chapter N
friendly labels), edit-note/delete actions branch by kind, and the
banner's export now covers all annotations. The archived ledger
(ListDeletedAnnotationsForBook) already tombstoned bookmarks equally
and Restore/Purge take annotation_type=bookmark — verified live:
delete → restore brings the row back with its note intact.
Volume: the dragger sits BELOW the mute icon in the panel (stacked
flyout shape) with the horizontal slider from the last round.
Badges: the real fix for comics/manga — format_group only knows
'comic_archive' (manga_type is unreliable across sources), so the
comic/manga split is resolved from the LIBRARY type server-side.
handlers.MediaBadgeKind + BookInfo.Badge at every construction site
(dashboard SSR + sections API, collections, series, and the bookshelf
search grid — which was also missing FormatGroup entirely, so no card
ever badged there). Both renderers (templ + dashboard.ts) switch on
the resolved badge. Verified: All-Books grid renders 2 Comic + 3
Manga badges, dashboard the audiobook one.
Infra: header.templ main.js src gains ?v=<stamp> — browsers keyed an
intermediate bundle by URL and served it across rebuilds (the stale
'Add note' the user saw); bump the stamp when JS stale-shadows.
A2a (metadata round): narrators TEXT[] + narrators_search (tags
pattern: accumulator + GIN pairs) + abridged bool on media_items;
scanner fills from the Libation/ABS sidecar (narrators verbatim,
abridged flag) with audio-payload promotion; editorial fields ride
the real columns while audio_metadata stays scan-derived (duration/
codec/chapters). Detail/edit/save paths carry the fields; the web
edit dialog grows an Audio section (Narrator(s) comma-input, Abridged
toggle, scanned duration read-only); search weights + narrator filter
(word_similarity over narrators_search) + narrator autocomplete added.
A2c (per-media-type wording): copy helpers keyed on format_group —
Listen Now, Mark Complete/Not Complete, Listening Progress, Your
Listening Stats, Time Listening, Time listened (pages tile swapped for
audio), Position H:MM:SS / total H:MM:SS on detail + progress pages
(duration parsed from audio_metadata). Dashboard: system collections
swap to Continue Listening / Recently Listened at DISPLAY time when
the audiobooks library is selected (stored names unchanged; restores
keep working); week card = Listening Time / Sessions / Books Touched;
cover-corner badges identify audiobook (headphones) / comic / manga
cards on mixed grids — books stay unbadged; hover action icon follows
the media type.
A2b (player): the scrubber binds value to the playhead with an explicit
scrub guard (the handle previously froze at its initial max); header
row = reader-style back arrow + Chapter n/total launcher; chapters
open in a floating top-down drawer (current chapter highlighted, page
never shifts); Skip/Speed rows share one label-left layout; honest
wall-clock session accounting (playing-only deltas, paused flush).
A2d (bookmarks): note-bearing time bookmarks ride the existing
media_bookmarks substrate (notes column + sync-aware service already
there) — audio locator convention audio:<seconds>, chapter_number
filled, auto-title '<chapter> — H:MM:SS', capture popover with optional
note, drawer with jump/edit-note/delete. Zero server changes needed.
Verified live on the Libation sample: narrators backfilled by rescan;
PUT edit persisted + overrides detected (narrators, abridged); force
rescan preserved the user edit; reset-to-scanned-of-defaults restored
truth; narrator filter/autocomplete scoped by library; dashboard
listsening wording + badges render; bookmark save with note deduped to
one row.
Phase 1 (foundation): 'audiobooks' library type seeded (schema + DB);
audio_metadata JSONB on media_items (the kobo_metadata precedent —
only audio paths read it); reading_progress.audio_offset_seconds for
the honest {chapter, offset} locator; extensions/mimes registered for
ALL breadth formats (m4b/m4a/mp3/flac — deliberate per the DOC/LIT
doctrine, m4b scans richly first); classifyFormatGroup 'audiobook'
case (gotcha 39 recomputes on rescan); Go mime table extended at boot
so media elements accept the streams; ffmpeg/ffprobe in the image
(8.1.2 — also the xHE-AAC identification tripwire).
Phase 2 (ingest): sidecar-first (Libation/ABS metadata.json —
narrators/ASIN/curated chapters), ffprobe fallback/filler (duration,
bitrate, codec/profile, embedded chapters). Fixed three real
parser brittlenesses the Libation sidecar exposed: series accepts
string OR array-of-objects, publishedYear accepts number OR string,
ffprobe chapter timings accept number OR string (flexFloat). Folder-
structure fallback now runs on the UPDATE path too (rescans were
clearing sidecar-absent series/author); audio facts re-derive on
every rescan (gotcha 39).
Phase 3 (playback + sync): GET /api/media-items/:id/chapters (the
format-abstraction list); streaming rides the existing cookie-authed
Range /uploads path; PUT progress carries audio_offset_seconds +
elapsed_seconds (honest wall-clock playing time) — the reading-
history capture prefers player-reported elapsed over the inter-push
delta, same 2h ceiling; reading_mode='audio' marks the row.
Phase 4 (web player): dedicated audio reader page (foliate not
involved) — cover, chapter title, whole-book scrubber, skip ± (chips
1/5/10/15/30s, default 10s), speed (0.75×–2×), chapter list with
jump-to, Media Session lockscreen/notification transport, periodic
playing-only progress pushes, position restore on open.
Verified live: ingest of the Libation sample (Gardens of the Moon —
title/author/series/cover/date from sidecar+folders, 34 chapters,
26h duration, aac/LC); chapters endpoint; the reader route serving
the player; playback with position restore (0:57 round-trip);
progress rows in audio mode; wall-clock session rows in
reading_history.
Extends the device-authed koreader surface with the handoff item 4
Batch B contract:
- GET/POST /api/sync/koreader/rating — the device mirrors the .sdr
summary's 5-star rating against the 1-10 server scale (POST 0 clears).
- GET/POST /api/sync/koreader/review — raw-markdown review mirror
(summary.note); empty body clears, matching the first-party PUT.
- POST /api/sync/koreader/stats — the device pushes its CUMULATIVE
total reading seconds; the server watermark (highest total ever
converted, read from the koreader sessions' own device_metadata)
converts only the delta into a reading_history session attributed
to the koreader device. Reinstalls cannot double-count; sub-minute
deltas ride forward (no session, no watermark movement); a lagging
device clamps to zero instead of dragging the total back.
Book identity resolves through the existing priority chain (uuid →
sha256 → alias → title/author). GET binding uses query tags; POST
endpoints bind the ref+payload once (an earlier resolve-then-bind
draft drained the body).
Live-verified against the dev server with the koreader device token:
rating set/get/clear, review mirror, and the stats sequence
900→session, 960→delta session, 970→rides forward, 500→clamps; the
book's stats card and the By Device panel both pick up the split.
media_reviews table mirrors media_ratings' shape (unique per user+item,
cascade deletes, updated_at for LWW) but standalone: a review without a
rating must work. Body is RAW MARKDOWN — storage and every payload carry
the source; rendering happens only at display sites (same doctrine as
notes, same sanitization pipeline: markdown-it + DOMPurify).
API: GET/PUT/DELETE /api/media-items/:id/review (PUT upserts; an empty
body deletes — clearing via an editor is a delete, not an empty row).
Book-detail page payload now carries the review beside the rating.
Page: the review renders above the synopsis as a scrolling card (same
max-height box as the summary), hydrated client-side through
renderNoteHtml; 'Write review'/'Edit review' sits next to the rating
stars and opens a raw-markdown editor dialog (save / delete / cancel).
sqlc regenerated; schema applied to the dev DB. Live-verified: upsert,
get, empty-delete, page markup, and the app round-trip against the dev
server.
Bug: the admin 'Scanning Libraries' widget fed the server's 0-1
progress straight into percent strings and bar widths, so the overall
and per-library percentages capped at '1%' and the bars never filled.
Both now scale by 100 (matching the setup page, which was already
correct).
Bug: scan state only reached clients via websocket events, so a page
opened (or refreshed) mid-scan showed nothing until the next tick —
and the admin widget never re-attached at all. The worker now exposes
active jobs (with their library_id so rows keep their names), served
as GET /api/scanner/active (admin, re-attaches the widget + polling)
and GET /api/scanner/active-summary (any user, restores the sidebar
indicator on load). Idle hides both, exactly as before.
Annoyance: the sidebar's 'Scanning… N%' was a block element between
the logo and the nav, shoving every menu entry down while a scan ran.
The percent now lives inside the fixed-height logo row next to the
spinner (spinner last, so growing digits extend leftward); nothing
below it moves.
Three dashboard complaints, one build:
- The widget collapses to a single summary row (This Week · minutes ·
pages · sessions · books · streak · Full stats) on every screen size
and expands to the full grid via a details/summary toggle — the
carousel sections start immediately below it again.
- The 'Your Library' heading moved into the library switcher subbar
(LibrarySwitcher gains an optional title param; collections/series
keep the plain 'Library' label). The standalone heading block is
gone.
- Root-caused the JS-switch vanishing: renderDashboardCollections
replaces ALL of #collections-container's innerHTML, so anything
server-rendered inside it (title, widget) died on the first library
switch. The widget now re-fetches ITSELF: a /dashboard/week-widget
fragment (same resolveLibrary family-lens rules as the page) is
pulled by htmx whenever the switcher dispatches week-refresh —
carrying the current #library-select value — so the widget both
survives switches and RE-SCOPES to the switched library. Initial SSR
renders the same scope the sections use.
ThisWeek (and the streak calendar) gained an optional library filter.
Tests: TestReadingStatsWeekLibraryScope — per-library totals, scoped
streaks, invisible-library isolation, malformed id 400. Full suite
PASS.
- Attribution: registered clients may now send their device token in
X-Device-Token on PUT /api/media-items/:id/progress (the user JWT
stays primary). The token must map to a device OWNED BY the caller —
foreign/invalid/absent tokens soft-fail to plain web attribution so a
stats decoration can never break a sync. Attributed pushes capture
device_id (FK-safe), device_metadata.source = the registered device
name ('Pixel 9', 'Kindle Paperwhite'), and last_sync surfaces show
that device. Side effect worth having: app-vs-web pushes used to be
LWW-invisible (both sources read 'web'); they now surface as
legitimate cross-source progress conflicts when they diverge.
- /stats gains a By Device panel alongside By Library (grid 3-wide):
reading minutes + sessions + books per registered device, plus one
Web / unsynced bucket for device-less pushes so the panel always
reconciles with the headline totals. Range-aware, family-lens
scoped; renderBars shared between panels in stats.ts.
- The device identity IS the registered platform: devices rows already
carry the name/type the client registered (app sends Build.MODEL +
'mobile' at /api/devices/register, KOReader/Kobo likewise) — no new
identity plumbing on the server.
Tests: capture attributed to the device (row device_id + source
metadata + last_sync_source), foreign/nonexistent tokens soft-fail,
device buckets reconcile (80 attributed + 75 unsynced = 155 total).
Full suite PASS.
The design parked in the handoff (multi-tier stats off reading_history;
bookorbit-style in data, not looks) is in. The real prerequisite work
was data: reading_history existed but NOTHING wrote to it — the whole
stats feature was sitting on an empty table.
- CAPTURE (no client changes, no repo-boundary crossings): every
successful progress write through SaveProgress (KOReader + app + web
+ Kobo + offline queue — the one chokepoint) now records one
reading_history session row: span = the interval since the PREVIOUS
write instant, pages = positive page delta when both sides know the
page index, marker rows for first contact. Spans over 2h are kept as
zero-second markers (overnight suspend→resume, offline flushes must
not inflate totals) — the honest 'sync cadence, not a stopwatch'
caveat stays. device_id only when a REAL device row matches (the
web/app progress path historically stuffs the USER id into DeviceID
— probe instead of trusting it; FK violation would silently drop
every capture). Source preserved in device_metadata.
- API: /api/stats/summary?range=7|30|90|year (totals, zero-filled
daily arrays, streaks, most-active day/weekday, top books, library
breakdown — one call), /api/stats/this-week (widget: rolling 7d vs
previous + streak), /api/stats/books/:id (lifetime + recent
sessions). All bucketed in the user's timezone; finished ≡ any push
≥0.995; visible-library lens enforced at source. Legacy
/api/analytics/* kept untouched. Range windows anchor end-of-day to
the START of today (blasting the clock time into tomorrow resolved
'today' a calendar day late — found by the streak test).
- /stats page (sidebar Stats between Progress and Devices, previously
/analytics was an orphan): range selector, five stat cards, two
Chart.js v4 charts (daily minutes bar, pages/day line — chunked via
dynamic import so main.js stays untouched), per-library breakdown,
most-read list, empty-state explainer. /analytics now 301s to /stats.
The orphaned analytics.templ + analytics.ts removed (their containers
and the page JSON never matched; the page was astub).
- Dashboard 'This Week' widget (server-rendered, suppressed on service
failure) and the book detail 'Your Reading Stats' card (lifetime
totals of the viewer's own sessions) complete the tiers: per-user
default, per-library breakdown inline, per-book on the page,
per-system/admin aggregate parked as before.
Tests: statistics_test.go (capture semantics at the chokepoint,
tz/day buckets, zero-fill, streak math, finished detection, range
validation, auth gate, widget, per-book lifetime). Full suite PASS.
Type is DERIVED, not stored: a saved ruleset makes a collection SMART,
no rules = STATIC (stored collection_items links, unchanged behavior).
No schema change — collections.auto_assign_rules JSONB already carried
the config; user's decision 2026-10-14 (handoff: live-at-render, not
refresh-on-scan; create-time Static/Smart toggle; v1 NO hybrid).
- Rule gains an enabled flag (absent = enabled, legacy rules unaffected).
- Rule CRUD: POST/PUT(DELETE) /api/collections/:id/rules[/:ruleId] — the
survivor dead effort of the rules-builder UI finally has real
endpoints (rules were never editable after creation; nothing ever
re-evaluated them). First rule converts a user collection to smart;
removing the last reverts it to static. UpdateCollectionRulesJSON
edits ONLY the rules doc so rule CRUD cannot clobber metadata.
- Live evaluation: candidates from GetVisibleLiveMediaForRules —
visible-library set (family lens enforced at source) + archived IS
NULL AND missing_scan_count = 0 — any enabled rule matches, title
order. Zero refresh hooks by design: always current by construction.
- Membership mutations (add/remove/bulk-add) are 400 on smart — v1 no
hybrid; collection_items.excluded stays the reserved future hook.
- Rules builder page (/collections/:id/rules) first time ever routable;
templ whole-attribute interpolation ("{ expr }" inside a quoted attr
value is literal text, as theme-{ user.Theme } silently proved).
- UI: create modal Static/⚡Smart cards (smart create redirects into the
rules builder), ⚡ badge + real per-card counts on the grid (smart =
live matches, static = stored-link count), smart detail renders
matched books with no hand-pick actions + an Edit Rules button.
- API: GET /:id gains is_smart + live books; GET /: keeps the Android
app's default exclusion, include_auto=true surfaces smart rows with
live counts; TestRules previews against the SAME candidates the live
render uses (visible + live, was raw first-1000).
- FIXED pre-existing regression from 7ce7321: static collection detail
with NO library_id returned zero books — QueryType != '' caught user
rows ('filter' default) and routed them to dashboard sections that
never contain them; now only real system query types go there
(handlers + SSR). App-visible symptom: /api/collections/:id without
library_id came back books:[] for every static collection.
Tests: collections_smart_test.go (conversion, disabled rules, mutation
guards, revert-to-static, validation, system-collection prohibition,
include_auto semantics). Full offline integration suite PASS.
GET /api/auth/password-rules (public, pre-auth) exposes the ACTIVE
complexity config from the settings registry; password_validation.ts
now fetches it and generates the checklist rows per active rule —
inactive requirements no longer linger as fake tasks, and the min
length mirrors the admin's setting instead of a hardcoded 8.
Surfaces: register, setup, admin create-user (Users page), profile
password change (self + admin variants, now gated on checklist pass;
misleading minlength=6 dropped — the server was always the enforcer).
Verified live: flip min_length=12 / number+special off via the
settings API → endpoint + register/profile/admin checklists re-render
row-for-row; enforcement follows (short rejected, rule-valid accepted
without digits/specials); restore defaults → checklists return.
Three changes from the network-drive audit:
1. Delete-ordering fix. Deleting a media item cascades its annotations,
so the delete must never proceed while the item's library storage is
unreachable — the DB row died, the file survived as an orphan, and
the next scan re-imported it bare (annotations gone forever).
resolveMediaStorage stats the library's folders: reachable → delete
proceeds and the file is removed by ABSOLUTE path (the old code
passed a library-relative path to os.Remove, which never resolved —
file deletion has been a silent no-op); every folder unreachable →
409 with 'storage unreachable', row and annotations intact.
2. Latency: resolveLibrary stats every library folder on every SSR
render; folder reachability is now cached with a 30s TTL.
3. README: Network Storage section documenting the outage hysteresis
(mark → archive → auto-restore), the 90-day retention purge
(ARCHIVE_RETENTION_DAYS, 0 = never), refused deletes while storage
is down, and watch mode not firing on network mounts.
Integration tests: storage-down bulk delete refuses with the row
intact; real-file delete removes the file from disk.
The admin/user profile modal PUT succeeded but the response was a bare
JSON body swapped into a small div at the bottom of the still-open
modal — no close, no toast, nothing moved. It read as 'the button did
nothing' even though the save landed. HX-Request calls now answer with
HX-Refresh (re-render shows the saved values everywhere and closes the
modal) plus a visible confirmation. API callers keep the JSON body.
Silent-failure display already exists site-wide (toast.ts listens for
htmx:responseError), so failures were never the silent case.
The audit showed only 5 of ~11 time-display surfaces converted to the
user's timezone; everything else rendered pgx-decoded UTC. The server
owns conversion — clients (web/app/KOReader) receive display-ready
strings or machine-readable instants and never convert.
- Progress page: LastUpdated formatted in the requesting user's tz
(was raw UTC).
- Analytics: date-range defaults, day buckets, most-active-day, device
last-sync, and popular-book last-read all computed in the user's tz —
evening sessions no longer land on the next UTC day. Also fixed the
end-date boundary: the query bounds with <= and the end bound was
midnight, so the end date's own sessions (including 'today') never
appeared.
- Queue page: new created_at_display field (admin tz) rendered by the
template; created_at stays RFC3339 for API consumers.
- Admin hash-conflicts SSR bridge: detected date rendered in the
admin's tz.
- OPDS: .UTC() before the Z-suffixed feed timestamps (honesty fix —
correct today only because the session zone happens to be UTC).
- KOReader annotation datetimes deliberately untouched (device owns its
display).
default_timezone existed as a system setting (admin-editable, exposed
via SettingsRegistry.DefaultTimezone) but nothing read it at user
creation — the CreateUser query didn't even carry the column, so every
new user got the DB default UTC regardless. Both creation paths
(public register + admin create) now seed the user's timezone from the
setting, validated with time.LoadLocation and falling back to UTC when
unwired/empty/invalid.
Test-infra fixes surfaced by the run: TestUpdateProfileSelfEdit was
self-editing the SHARED admin (renaming it past the harness cleanup's
preserve-list, leaking a second admin row that broke the last-admin
tests) — it now edits a dedicated user with a unique name; the
last-admin tests get an ensureSingleAdmin precondition helper; the
websocket collection name is run-unique (non-test names persist in the
shared DB and UNIQUE(user_id,name) 409s on rerun).
The family-lens hardening: library_visibility was only enforced where
file bytes flowed (download/reader/OPDS) — metadata surfaces leaked
hidden libraries to any authenticated account, and registration was
open by default with no admin-side user creation.
- Series are grouped PER LIBRARY now (same name in two libraries =
two entries: manga adaptations / LN splits no longer merge);
/api/series aggregates only the requester's VISIBLE libraries and
/api/series/books REQUIRES a visible library_id (breaking contract;
web + app clients updated in the same change).
- /api/media-items, item detail, and the reader content endpoints
(chapters/panels/outline/THUMBNAILS — page images leaked unguarded
next to a gated GetPage) all enforce the visible set; resolveLibrary
no longer honors a hand-typed library_id outside the visible set.
- Annotation/note/bookmark mutations (update/delete/tombstone, sync
LWW paths incl. conflict resolution) carry ownership predicates:
another account's bare row id is a 404, not a silent delete.
- Admin user creation: POST /api/auth/users (AdminMiddleware) + a
Create User card in the admin Users UI — accounts stay creatable
with self-service registration OFF, which is now the schema default
(fresh installs; the first-admin bootstrap exception is unchanged).
- Tests: series contract re-pinned (per-library, visibility 403s),
new ownership + admin-create integration suites, OPDS all=1 test
now passes an explicit page size (shared-DB growth made the fresh
books fall off page 1), device-cap helper switched to the admin
endpoint.
KOReader highlights converted to wrong or unresolvable web CFIs
whenever a selection crossed text-node boundaries. Three compounding
defects: ParseCREXPointer discarded the text()[N] node index and
applied .offset as a global block offset; the end anchor converted
with an empty context (blind structural landing, zero verification);
and the block-containment check accepted wrong-node landings because
a selection's text always occurs somewhere in its own block. pos0 and
pos1 were converted independently, yielding mismatched start/end base
paths and backwards ranges - highlights stored but never painted, or
painted at the wrong location, in the web reader.
Add ConvertHighlightRange: both endpoints are anchored by locating the
selection text in the flattened document text (whitespace-normalized,
crossing block boundaries), with the device locators demoted to
disambiguation hints. A single document match yields a same-spine,
ordered start/end pair by construction. Falls back to the legacy
per-endpoint path when the text cannot be located, so behavior is
never worse than before.
Also fix the shared flatten stream: zero-width format characters
(ZWSP/ZWNJ/BOM) and whitespace-only inline spans (<span> </span>)
now act as word separators instead of fusing adjacent words, which
prevented text matching in real books.
Verified against 28 real device highlights from production: converter
output now matches the text-search oracle on all of them (previously
8/28 correct, 18 with mismatched base paths, 2 backwards).
Handler.UpdateUniversalProgress (handlers/progress.go) had no route
registration anywhere (router/progress.go only wires the history route
to the scanner handler) and no callers — a legacy progress-write path
that bypassed the verify/heal pipeline entirely. Deleting it removes
the last bypass-shaped writer that could poison progress rows without
verification; conflict-resolution apply remains the only deliberate
direct write (user-approved winner rows).
Pure deletion: 115 lines, no route, no callers, no behavior change.
The verify/heal pipeline verified only INBOUND context-bearing writes;
stored-row coherence was never enforced. SaveProgress seeds missing
fields from the existing row, so every percentage-only write (KOReader
low-confidence conversion, KOReader no-pointer payloads, Kobo markup
ReadingSync / analytics / sync-from-server, locator-less queue entries)
silently re-paired its NEW percentage with the PREVIOUS writer's stale
CFI+context — the incoherent row shape that fed the client-side
far-anchor restore floor (and, via the read-side anchor handles, could
even hand a first-party client the OLD position's scroll address).
KOReader's converter already declared the right intent ("storing
percentage only — the row's percentage restores approximately"); the
seeding betrayed it. SaveProgressRequest gains ClearLocator, set by the
percentage-only call sites, and the seeding rule honors it: stale
epubcfi/context/character_offset clear (the upsert assigns columns
directly, so NULL sticks) unless the request carries its own. A
request-carried context survives so the verify/heal pass can still mint
a fresh anchor from it; the HTTP merge path (field-absent PUTs
preserve) and all anchor-bearing writes are untouched. The fixed-layout
branch already cleared locators unconditionally — unchanged.
Consumer audit: percentage — the only thing every sync round-trip
exchanges — is untouched on all paths; KOReader pulls on locator-less
rows already support the nil-CFI shape (builder guards
progress.Epubcfi.Valid); first-party restores go percentage-first
instead of floor-fighting a stale anchor.
Tests: TestProgressWeb_LocatorCoherence pins clear-on-declared,
request-carried-context survival, and merge-path preservation. Full
suite with the change: 218 groups, zero failures.
The create endpoint shared the scanner's upsert (ON CONFLICT
(library_id, file_path) DO UPDATE SET updated_at = NOW() RETURNING *) —
so a POST naming an already-registered file returned 201 CREATED with
the OLD row while silently discarding everything the caller submitted.
Nothing depended on that lie: the app only GETs/PUTs media items, the
web only searches/purges/deletes, KOReader never calls it — the
endpoint's create path had zero production callers.
Split the API path from the scanner's idempotent upsert, which is kept
verbatim in CreateMediaItem (re-scan dedup remains its safety net):
- CreateMediaItemIfNew: same insert with ON CONFLICT DO NOTHING — the
loser of a create race writes nothing and returns no row.
- GetMediaItemByLibraryAndPath: fetch the existing row for the
duplicate response.
- Handler branch: on pgx.ErrNoRows → 200 OK with
{"duplicate": true, "message": ..., "item": <existing row>} instead
of a 201 pretending it created something. Real creates still answer
201 with the item as before.
Bulk-import shape (the reason for 200+flag over 409): a re-run after a
mid-import failure is harmless and countable — each response tells the
caller exactly which of the two things happened.
Verified by the full integration suite: 217 groups pass, zero failures,
including a new pin of the duplicate contract (same file_path twice in
one library → 201 then 200/duplicate:true, second payload not
overwritten).
Born dead in e7ce0cb (2026-04-03, backend reader infrastructure):
the "Phase 2.5" plan assumed MOBI/AZW3/DOCX/RTF would need server-side
parsing, so ParseEbook was written as the endpoint that would do it —
but it only ever returned 501 "not yet implemented", and no route in
router/reader.go ever pointed at it. No caller in Go, templates, or
web JS/TS ever existed either.
The eventual architecture never needed it: the web reader hands the
raw file to foliate-js and parses client-side (MOBI renders via the
fork's mobi.js — verified live; DOCX via mammoth; RTF/PDB shipped).
By now it was also stale — it 501'd DOCX/RTF as "not yet implemented"
while both work, and offered an AZW3 path that was dropped entirely.
Pure deletion: no route registration, no references, no tests, no
behavior change. go build ./... clean.
The old model hard-deleted system collections and re-created them at
the tail, resurrecting stale dedups and breaking ordering. Redesign
(user-directed B-final):
- schema.sql: collections.deleted_at TIMESTAMPTZ (idempotent ALTER —
the whole schema is re-applied on every boot via executeSchema).
- queries.sql: every collection list/read query filters
deleted_at IS NULL; new GetSystemCollectionByQueryType :one
(unfiltered, sees tombstones), TombstoneSystemCollection :exec,
ClearCollectionItems :exec, RestoreSystemCollectionRow :one
(in-place metadata reset + un-tombstone; collections have no
updated_at column — ResetUpdatedAt was a 500 caught by tests).
sqlc regenerated.
- collection_service.DeleteCollection: system collections are
tombstoned (preserving position/ownership); user collections keep
the hard delete. No ErrNoRows to the client (delete is idempotent).
- dashboard_service.RestoreSystemCollection: resolve by query_type;
found → in-place restore (ClearCollectionItems + metadata reset,
un-tombstoned); missing → provision fresh. "Want to Read"/etc.
stay at their home positions forever.
- handlers/collections.CreateCollection: name-taken is 409, not 500.
- handlers/auth.CreateDefaultCollectionsForUser: idempotent by
(user_id, query_type) — update-in-place when a row for that query
type exists (heals renames/tombstones), insert when missing; this
also fixes the duplicate-system-collections regression at source.
The dashboard's All Libraries view could render sections across every
library but could not persist its own preferences: the PUT required a
UUID library_id, and the prefs GET-without-library_id matched an
arbitrary library's row through a loose NULL OR-clause.
Storage: a dedicated user_dashboard_preferences_global table (one row
per user, user_id PK → users CASCADE) instead of a NULL/sentinel
library_id — chosen over both because (a) the column's FK to libraries
rules out a sentinel UUID without dropping the constraint or minting a
fake library row, and (b) intent becomes structural: only the explicit
global path writes that table, so a buggy or outside-the-API write can
never masquerade as global. System and user collections are
user-scoped, so the order/hidden arrays are library-independent and
valid in this scope.
API: PUT /api/dashboard/preferences binds library_id as *string —
JSON null/absent = global, empty string = 400 (a lost value must stay
loud, not silently clobber the global scope), valid UUID = per-library
(unchanged). GET without library_id now returns the global row instead
of an arbitrary one; the per-library query's dead NULL OR-arm is
removed. GET /sections keeps requiring library_id for per-library
views and aggregates across libraries when absent (unchanged).
Tests: global save/update-dedupe/read round-trip, empty-string 400,
global-vs-per-library independence. Two suite failures are the
handoff's documented pre-existing rot (duplicate system collections in
the dev DB; a sections test expecting a 400 the handler never
implemented) — reproduced and confirmed unrelated to this change.
Live-verified on the dev stack: null save 200 x2 with a single row
updated, empty string 400, global read carries the second save.
One boolean (Security > Public Registration) switches the whole public
signup lifecycle: front page + login links, the logged-out sidebar's
Create-an-account (via a package-level templates hook so shared page
templates keep their signatures), GET /register -> 302 /login, and an
early 403 on POST /api/auth/register. First-user exception preserved:
zero admins keeps every route and link reachable for bootstrapping —
same 'users exist' reasoning as the setup gate. Admin user creation is
unaffected by design.
Live E2E verified: flag off hides all surfaces and blocks the POST
(403), flag on restores them (201); the admin UI renders the row
automatically.
The 10k validator cap rejected legitimate long-form notes — a
scholarly reading note with quoted passages and footnotes lands
around 10.1k chars and failed at save with the raw validator error.
Raise the cap to 100,000 on all six annotation text fields in the
media handler request structs:
- CreateMediaNoteRequest / UpdateMediaNoteRequest Content
- CreateMediaHighlightRequest / UpdateMediaHighlightRequest NoteText
- CreateMediaBookmarkRequest / UpdateMediaBookmarkRequest Notes
No other layer changes: the media_notes / media_highlights columns
are unbounded TEXT, and the KOReader + websocket sync paths never
had a length cap, so the REST API now matches the rest of the
system instead of being the strictest gate.
New integration test pins the boundary: 50k and 100k-char notes
return 201, 100,001 chars returns 400.
GetProfile returned the theme from the JWT-claims context stub, which
carries only id/email/username/role — user.Theme was always empty, so
the profile response omitted the field and the app's account-theme
read-back (SERVER sync mode) could never see the stored theme. GetProfile
now loads the full user row by id (h.db.GetUser).
The Android app's app-chrome theme (UX pass item 6) reads the account
theme from GET /api/auth/profile and re-applies it at startup in
SERVER sync mode. PUT /api/auth/theme already persisted the value;
this completes the round trip on the profile read.
Adds publisher_filter to SearchMediaItems/SearchMediaItemsUnified
(fuzzy word_similarity against mi.publisher, mirroring genre_filter),
plumbs it through services.SearchParams and the search handler, and
extends the GREATEST relevance ranking to include publisher matches.
Serves the app's author/publisher/genre/tag click-through browses
(UX pass item 5 — publisher was the only facet without a server
filter).
Offset currency policy, now explicit: EPUB CFI terminals, CRE text()
offsets and the served char_offset handle are UTF-16 code units (the
EPUB CFI spec, and what foliate/readium/KOReader/Kobo clients actually
observe), while internal arithmetic — the book-wide character_offset
column and percentage fractions — stays rune-based, consistent with
TotalCharacters. For all-BMP books the currencies are identical, so no
stored value changes; astral-plane text (emoji, rare CJK) no longer
drifts.
Boundaries converted: resolveCFIToNode interprets incoming CFI terminal
offsets as UTF-16; textNodeAtUTF16Offset (née textNodeAtRuneOffset)
interprets CRE text() offsets as UTF-16; buildCFI and buildCREXPointer
emit UTF-16 terminals; blockCharOffset (the served char_offset) is
UTF-16.
Also fixes two character_offset column defects: heals wrote a BLOCK-
relative offset into the book-wide column, and verified-but-unhealed
saves (e.g. KOReader pushes) never refreshed it, leaving it stale
behind the anchor. VerifyProgressAnchor now returns the verified book-
wide rune offset and SaveProgress refreshes the column on every
verified save.
Tests: astral currency round trip (offset after an emoji must shift by
one unit between currencies, in both heal and exact-verify directions)
and book-offset ordering. The cmd/server/tests integration harness
failures under docker (library folder 400 during setup) reproduce on
the pre-change tree and are unrelated.
Progress submissions now carry (percentage, context_text, epubcfi) and
the server becomes the position authority:
- VerifyProgressAnchor resolves the submitted standard CFI against the
book's own XHTML, extracts the text at the anchor, and cross-checks it
with the submitted context_text. A mismatch heals the anchor by text
search (percentage disambiguates repeats) instead of storing a bad
position.
- The anchor's block element is derived as a cssSelector plus a block-
relative character offset, and served on progress GET alongside the
anchor document's href — readium-native handles that let clients
re-open a book without parsing CFIs themselves.
- context_text-only submissions (no CFI — the dumb-client tier) are
anchored structurally from the context text.
Motivation: cross-client progress sync (web foliate CFIs, KOReader CRE
xpointers, readium-native apps) previously trusted each client's own
locator math; the app's EPUB restore drifted ±pages because readium's
paginator does not lay out far-from-viewport columns and the foliate-
ported CFI walk ran against readium's mutated WebView DOM. Server-side
verification heals both classes at ingest.
The device catalog flattened every visible library into one list, so
duplicate copies of the same book each got an entry and search had no
library context. The root feed is now a standard OPDS 1.1 navigation feed
that mirrors the web UI's library model:
- Root /catalog: an "All Books" entry first (the previous flat cross-
library behavior, also still served at ?all=1 for clients that want
the single flat list), followed by one folder entry per visible
library with live book counts from GetVisibleLibraryMediaCounts.
- New GET /library/:libraryId/catalog: acquisition feed scoped to one
visible library (403 when the device owner cannot see it), paginated,
with an up-link back to the root.
- Scoped search: each library feed's rel="search" OpenSearch template
pins &library_id=<id>, so opening search from inside a library folder
searches only that library — clients substitute only {searchTerms},
so no client-side changes are required. Root search stays global.
- Global search entries now carry the owning library as a category (and
as a fallback summary when the book has no description), so duplicate
copies are distinguishable in unscoped result lists.
The acquisition entry builder is extracted into addAcquisitionEntries and
shared by the all-books and per-library feeds. bookhoard.koplugin needs
no changes: it only registers the root URL, and KOReader's stock OPDS
client renders navigation feeds natively.
Tests: TestOPDSLibraryFolders covers the nav-feed shape, flat ?all=1
mode, scoped catalog isolation, scoped/global search behavior, and 403s
for libraries outside the device's visibility. Library names avoid the
word "test" on purpose — setupDeviceTest re-runs setupTestServer's
setup-time cleanup, which deletes every library whose name contains it.
Covers the copy-then-delete-later workflow: a user copies books to a new
library, deletes the originals, and ends up with an active copy in the new
library plus an archived twin holding the real reading history. Until now
those twins could only be purged (destroying the history) or restored
(showing a permanently broken entry).
- POST /api/media-items/:id/merge (admin only, body {target_id}):
validates the source is archived/missing, the target is active, and
both share the same file_sha256; then re-parents every child row onto
the target via the existing reparent_media_item_children function (the
same machinery as hash-conflict resolution) and deletes the source row.
Per-user collisions keep the active copy's data, mirroring that flow.
Affected data: reading progress, speed, ratings, highlights, notes,
bookmarks (including tombstoned deleted-annotation history), formats,
collections, kobo shelves/entitlements, sync rows, panel data,
processing issues, and device aliases.
- GET /admin/archived: ListHiddenMediaItems' match_* columns surface each
row's best active twin; rows with a twin get a confirm-guarded "Merge"
button (data-merge-source/-target) next to Restore/Delete, wired in
web/src/admin.ts like the existing unarchive/delete handlers.
- templates.ArchivedItem gains MatchID/MatchTitle/MatchLibraryName;
frontend.go populates them from the listing row.
After a merge the archived row is gone, so the retention purge can never
destroy the merged data. Deleted-annotation tombstones carry over and
remain restorable from the target book's "recently deleted" history.
The mutex hardening added a fall-through 'pending' response path that
returned while still holding pendingMu — one status poll before approval
leaked the lock and every later register/approve/status request hung
forever (caught by TestDeviceRegistrationFlow's approve step hanging).
App reinstalls that preserve data (Android Studio installDebug over an
existing install) re-register with the same device_identifier, but the
devices row from the previous install still exists — device_identifier
is UNIQUE, so ApproveDevice's blind INSERT failed with a unique
violation and returned 500 'failed to create device' (reproduced via
curl: second approve with the same identifier = instant 500; the ~98s
in the original report was app-side retry/polling, not server wait).
ApproveDevice is now idempotent: look the device up by identifier
first; a row owned by the approving user gets its auth token rotated
via UpdateDeviceAuthToken (row id unchanged, so synced highlights/
bookmarks/progress anchored to it stay valid; fresh install = fresh
credentials, old token invalidated); a row owned by another user gets
409; unknown identifiers INSERT as before, with the 23505 race falling
through to the rotate path. DB failures are logged (they were silent).
Also guard the in-memory pendingRegistrations map with a mutex —
register/approve/reject/status/list all touch it from HTTP goroutines,
and a racing write is a Go runtime fatal, not an error. The approver's
credential publication and the status poller's approved-branch snapshot
now run under the lock so the token can never be read half-written.
Regression test: TestApproveDeviceReapprovalRotatesToken — register →
approve → re-register same identifier → approve (must be 200) → token
rotated, exactly one devices row, row carries the new token.
PDFs shelved in comics or manga libraries (scanned manga, official manga
PDFs with text layers) now get comic reader treatment: bookmarks only —
no text-selection highlights, no annotations, no in-book search. Items
in ebooks libraries are completely unaffected, and EPUBs keep their
existing behavior everywhere.
- handlers: new exported ShouldTreatAsComic(libraryType, formatGroup) —
true for fixed_layout/comic_archive in manga/comics libraries; the
reader JSON API also exposes it as treat_as_comic
- router: the SSR reader page fetches the library type and passes
TreatAsComic through ReaderMetadata into the init config
- reader: when treatAsComic is set, the PDF textLayer selection listener
(mouse and touch paths) never attaches so no highlight popover can
open; pointer mode is forced to pan (Smart/Text segment hidden);
search button and toggleSearch are disabled; previously-created PDF
highlights stop rendering
- bookmarks are unchanged (already page-index based for fixed layout),
and device sync / OPDS / format classification are untouched since
format_group stays fixed_layout
Admins need to see what the archive lifecycle is holding: a dedicated
/admin/archived page listing every hidden item (archived or still in
the missing grace window) with library, file path, status, and - when
retention is enabled - the exact date it will be permanently deleted
(archived_at + ARCHIVE_RETENTION_DAYS).
Per row: Restore (POST /api/media-items/:id/unarchive, clears the
archive state so it reappears; if the file is still gone the next scan
hides it again) and Delete (existing DELETE endpoint for single-row
purge with its reading history). Purge All Archived reuses the existing
bulk button. The library admin banner links to the page and keeps its
purge button; row actions use data attributes with delegated listeners
in admin.ts since this templ version has no JSFunctionCall helper.
Verified live: page 200 with purge dates shown, unarchive returned 204
and reset the row, per-item delete and bulk purge ({purged:1}) both
removed their rows with no leftovers.
Bulk escape hatch for archived rows (files missing from disk for 2+
scans) so a mass external deletion never has to wait out the retention
window or be clicked away row by row:
- POST /api/media-items/purge-archived (admin only) hard-deletes all
archived items and returns the purged count; reading history goes with
the rows, so the call is confirmed in the UI first.
- The library admin page shows an 'Archived items: N' card (only when
non-zero) with a Purge Archived Now button that calls the endpoint,
toasts the result, and reloads.
- Frontend admin JS exposes window.purgeArchivedItems following the
existing localStorage-bearer-token pattern.
John O'Keefe
john-okeefe