The identification queue for device files the sync cannot match is now
live end-to-end. Previously the whole feature was dark: unmatched device
files were silently dropped (losing their reading progress), the queue
table sat empty by construction, and the page template had no route.
- Ingest: the KOReader sync loop, on an unresolvable file, upserts
unlinked_books (one row per device + content id; re-syncs bump
last_seen_at and refresh device-reported metadata) instead of
dropping the file. Schema gains the (device_id, content_id) unique
index the upsert needs.
- Page: GET /unlinked-books routes the existing page template, listing
the caller's own unresolved rows (ListUnresolvedUnlinkedBooksForUser
— devices.user_id scoping; no cross-user leakage).
- Dashboard banner: a warning card above the shelves, rendered only
when the caller's queue is non-empty (hidden-unless-needed), linking
to the page.
- Resolve flow unchanged and already gated: MatchingHandler.LinkBook
verifies device ownership and target-item visibility (403s) and
creates the device file alias, so the next sync matches by path.
SyncHandler.LinkBook and Handler.LinkBook are unrouted dead handlers
— triage candidates, not part of the live flow.
The audiobook library type shipped backend-complete (LibraryTypeAudiobooks
constant, allowed extensions, MIME map, schema seed row, GET
/api/libraries/types) but neither create-library form offered it: the
admin Create Library modal and the setup wizard's add-library step both
hardcoded their <option> lists to the original three types
(ebooks/comics/manga), so an audiobook library could not be created from
the UI.
Both dropdowns now render from the same DB-backed library_types table
the create API resolves types against — a future type appears without
template edits (the exact drift the rendering-ownership rule warns
about):
- templates/utils.go: OrderedLibraryTypes pins the historical dropdown
order (ebooks, comics, manga, audiobooks) and appends unknown future
types by name; LibraryTypeDisplayName renders labels with a
capitalize-the-name fallback for types added without a label
- templates/admin_library.templ: new LibraryTypeOptions fragment used
by the modal's select; AdminLibrary takes the types list
- templates/setup.templ: Setup takes the types list, same fragment
- internal/router/helpers.go: libraryTypeOptionsFor fetches via
LibraryService.GetLibraryTypes, falling back to the built-in type
names on DB failure so the forms stay usable (CreateLibrary still
resolves the name against the DB, so the fallback is never
authoritative)
- internal/router/frontend.go, internal/router/setup.go: pass the list
into the two pages
Tests:
- templates/utils_test.go: pure unit tests for ordering + labels
- cmd/server/tests/admin_library_page_test.go: SSR regression —
/admin/library must offer all four library types, audiobooks exactly
once (options come from the DB loop, not a hardcode)
- BookMatch now carries title/author/cover (the fields the old client
renderer always assumed but the API never sent — cards would have
shown 'undefined'); filled during the family-lens pass in QueryBooks
- new partial route GET /unlinked-books/matches renders the scoped
match cascade server-side; searchMatches swaps it via htmx.ajax
instead of rebuilding cards from JSON
- NOTE: the unlinked-books surface is currently dark end-to-end — the
templ page has no route, the live linking.ts renderer targets a
container no template has, and its Link button calls a
window.showMatchModal that does not exist. This commit ships the
scoped backend + partial; wiring the UI is a product decision.
Where the browser supports it (Chrome 111+, Safari 18+, Firefox 139+),
switchWithTransition now uses the native View Transitions API: the old
shelves are snapshotted, the fetch + swap happen behind the dim, and the
two snapshots crossfade — no blank phase, no manual opacity
choreography. The dim + spinner is hidden inside the update so it blends
away as part of the transition. Browsers without the API keep the
existing dim + fade sequence unchanged. Zero dependencies — the
crossfade is the browser's own.
The switch overlay intended a 50% dim (bg-opacity-50) but set its
background with an inline style — Tailwind's bg-opacity variable does
not apply to inline styles, so the overlay rendered fully opaque. Every
library switch wiped the material area to a solid sheet (chrome stays
visible: the sidebar sits at z-60 above the overlay's z-50) and
hard-cut back when the overlay dropped — reading exactly as a new page
instead of a fade. The dashboard showed it worst because its switch
holds the overlay longest (sections fetch + awaited week-widget refresh
behind it).
- overlay background is now theme-aware translucent
(color-mix 55% bg-primary over transparent); the dead bg-opacity-50
class is gone
- the dim lifts before the fade-in starts, so the new content's fade
plays fully visible rather than being hard-cut mid-transition
/api/books/match returned media metadata from ALL libraries regardless
of visibility: the resolver lookups (GetMediaItemBySHA256 and friends)
are global, so match results leaked hidden-library titles, covers and
ids to any authenticated user, and LinkBook accepted a link target from
a hidden library. Both now enforce the family-lens rule:
- QueryBooks post-filters the cascade by the requester's visible-library
set (absent-from-results — a hidden library's book simply never
matches), before determineAction so the suggested action reflects the
filtered set. BookQueryRequest gains a server-side-only
VisibleLibraryIDs field; nil keeps legacy callers unfiltered.
- LinkBook verifies the device belongs to the caller
(DeviceExistsForUser) and the target item's library is visible —
403 otherwise. Linking was previously unscoped on both axes.
- GetUnlinkedBooks likewise verifies device ownership (its service is
still a stub, but the route no longer answers for other users'
devices).
Also: the collections books-grid partial uses a light loader —
requested library validated against the visible set directly instead of
the page-only library resolution (folder reachability, counts,
dropdown payload), removing the extra queries that made library
switching hold the spinner longer.
And the dashboard library switch now folds the This Week widget's
htmx refresh behind the spinner (refreshWeekWidget awaits the widget's
swap, with a safety timeout) instead of letting it pop in after the
fade-in — the two-stage update introduced with the widget's
library-scoped re-fetch.
Guardrail: the rendering lint now counts all innerHTML assignments (not
just template-literal ones) — the old pattern undercounted 20 files at
80 sites, including .join("") builders like renderDashboardCollections.
Baseline rebalanced accordingly.
They never did: filterConflicts/toggleAllConflicts/updateBulkActions
were referenced in the templ but defined nowhere, and the bulk buttons
called bulkResolve()/bulkDismiss() with no arguments against functions
expecting (strategy, conflictIds).
- conflict cards carry data-status; the status dropdown filters them
client-side (pure client state — allowed to be JS), applied on page
init and re-applied after every partial refresh, with a
'no conflicts match this filter' note when everything is filtered out
- select-all toggles only the visible cards; the bulk bar shows the
selected count and resolve/dismiss now gather the checked ids and the
chosen strategy (the bar's own buttons use resolveSelected()/
dismissSelected() wrappers)
- dead onchange= globals became Alpine @change handlers on the server-
rendered controls; partial swaps re-bind via Alpine.initTree (already
in refreshConflicts) and re-apply filter + bulk state
Visible change: the dropdown's stated default (Unresolved Only) now
actually applies on load, so resolved conflicts are hidden until the
filter is changed.
The library admin page carried an inline <script> block of global
functions (toggleLibraryPanel, openEditModal, openDeleteModal,
openFolderBrowser plus the delete-modal close listener). They move into
the admin Alpine component as typed methods; the page body gains
x-data="admin" and the call sites switch onclick= -> @click.
Behavior unchanged.
Guardrail: make lint-rendering (wired into make test) fails when a
web/src file builds more HTML template literals than its recorded
baseline (web/src/rendering-allowlist.txt, seeded at 18 sites across 9
files post-twin-deletion) or when a .templ file gains an inline script
block. theme_flash_guard.templ is the only sanctioned inline script;
web/src/reader is a sanctioned island.
theme.ts: Array.from(classList) — DOMTokenList spread needs
lib.dom.iterable, which the tsconfig does not include.
Both pages had a templ render (initial load) and a TypeScript twin
(innerHTML rebuild on refresh) of the same fragment. The twins are
deleted; the server-rendered markup is now the only markup.
- collections: the library switcher swaps in the new books-grid partial
(GET /collections/:id/books-grid) instead of re-rendering cards from
the /api/collections JSON. Book loading is extracted into
Config.loadCollectionBooks, shared verbatim by the page and the
partial (smart / dashboard-query / manual branches, family-lens
scoping via resolveLibrary + UserVisibleLibraryIDs) so the two cannot
drift.
- conflicts: resolve/dismiss refreshes swap the new list partial
(GET /conflicts/list) with the stats bar as an out-of-band swap,
instead of the TS twin — which rendered a degraded card (no
checkboxes, no timestamps), pointed at a window.showResolveModal that
does not exist, and updated stats via element ids the SSR never had.
- shims.d.ts: window.htmx typing (ajax/process) for TS callers.
Visibility note: the books-grid partial inherits its family-lens
scoping from the shared loader (identical to the detail page); a
dedicated hidden-library pin test for the partial route is still worth
adding in the next test pass.
Codifies the standing rule discussed for the templ/JS split: content
derived from server data is rendered by templ and refreshed via templ
partials (htmx.ajax), pure client-local state may render in TS/Alpine,
web/src is TypeScript-only, and inline <script> in templ files requires
a pre-paint justification (theme_flash_guard.templ only). Lists the
sanctioned JS islands (reader, Chart.js, toasts, password validation).
Also refreshes gotcha #1: the 'harmlessly masked' aside is stale — the
quoted-attribute sweep fixed every occurrence; the note now points at
templates.JSONArg for free text.
templ does not interpolate { expr } inside quoted attribute values — the
braces ship as literal text (AGENTS.md gotcha #1). Swept every templ
source for the pattern and fixed all of it:
Body theme classes: 22 page bodies rendered class="theme-{ user.Theme }"
literally, so every page first-painted the :root default (Tokyo Night)
and flashed when main.js applied the real theme. All now render
class={ "theme-" + user.Theme } server-side.
Attribute interpolations (26 across 8 files):
- api_explorer: initAPIExplorerDoc path/body/response and the Copy
Request/Response buttons (invalid JS, silently dead)
- collections: book-selection :checked/@change/@click ids and four
hx-get search URLs (collection_id={ collection.ID })
- conflicts: Resolve button id
- devices: Kobo sync-url/auth-token element ids (copy lookups only
worked because the id and the lookup were the same literal)
- unlinked_books: search-matches/manual-link clicks and container ids
- queue: status/priority badge classes never matched their CSS
- dashboard: Restore-section tooltip text
- collection_rules/collections: collection icon color
Free text goes through the new templates.JSONArg (json.Marshal) —
templ.JSONString in this templ version returns (string, error) and
cannot compose in attribute expressions; uuids via device.ID.String().
Includes regen-only drift from running templ generate (error-path
normalization) on files with no source change.
Expand the catalog from 11 themes (10 dark, 1 light) to 37 (24/13):
all 18 remaining Omarchy defaults (palettes from each theme's
colors.toml) plus 8 canonical light themes (Tokyo Night Day, Solarized
Light, One Light, Ayu Light, GitHub Light, Gruvbox Light, Everforest
Light, Kanagawa Lotus). Lupine leads the light list as the default
light theme; the app default stays tokyo-night.
Appearance panel: Dark/Light segmented filter (SSR-initialized from the
user's current theme), scrollable theme list, Bookshelf section pinned;
active row revealed on open. Light themes carry color-scheme: light and
the soft-shadow token set. Public-page theme selects get Dark/Light
optgroups. New moon/sun icons.
tailwind.config: safelist the new theme ids — Tailwind v3 purges custom
@layer base class selectors that appear in no scanned content file, so
an unsafelisted id silently ships without its CSS (found the hard way).
theme.ts: applyTheme now swaps only theme-* classes (changing theme
from the login select no longer wipes min-h-screen) and keeps the class
in sync on <html> for the new ThemeFlashGuard — a pre-paint inline
script on the four public pages that applies the saved theme before
first paint.
Kanagawa's UI accent uses crystalBlue (#7e9cd8) instead of Omarchy's
foreground-as-accent so links stay distinguishable from text.
Host-side test runs use CWD=cmd/server/tests, three levels below the repo
root; the location search only went up two, so it never found
database/schema/schema.sql from there and init() fatal-exited. That gap
was being papered over by an untracked local symlink
(cmd/server/tests/database -> ../../../database) hidden behind a bare
`database` line in .git/info/exclude — which also broke `git add` under
database/. Fix the search list instead and drop the filesystem trick:
no symlink to commit, no exclude hack, works on any platform.
Verified: contract pins green from cmd/server/tests with no symlink
present (init resolves via the new entry); parse test green.
audio_metadata (media_items) and audio_offset_seconds (reading_progress)
were declared only inside their CREATE TABLE blocks — fresh installs got
them, upgraded installs never did, because columns reach existing
databases only via the idempotent ALTER block that schema.sql re-runs on
every boot. On prod (v1.15.0) this 42703'd every media-existence check
and the archive pass (1817/1817 scan errors), and every progress save,
since the shared progress upsert names audio_offset_seconds.
Both ALTERs match the declared shapes exactly (nullable JSONB; INTEGER
CHECK (audio_offset_seconds >= 0)); idempotent, applied on every boot.
Verified against a scratch database built to the v1.14.0 shape: overlay
the patched schema -> both columns created; re-run (every boot) skips
cleanly; probes resolve reading_progress.audio_offset_seconds integer
and media_items.audio_metadata jsonb. Parse test and a real container
boot (38 tables, 6 functions verified) green.
Every web-API write endpoint hardcoded Source:"web", so the Android
app's own pushes landed with last_sync_source/last_modified_source
"web" — the app was indistinguishable from the web SPA in progress
conflicts, device_sync_data, and annotation bookkeeping. The app CAN
say who it is (bookmark create already carried origin → origin_source);
the server just ignored it for labeling.
All six write sites (progress PUT, bookmark/note/highlight create+PUT)
now parse an ADDITIVE optional "origin" key, normalized through one
gate: "android"/"ios" pass through, anything absent/unknown stays
"web" (the label keys conflict claims and device_sync_data — free-form
client strings would pollute them). Precedence on progress: a paired
X-Device-Token still outranks origin. Bookmark origin_source now rides
the same gate. The web SPA sends no origin — its rows are unchanged;
KOReader/Kobo handlers keep their own labels.
Wire discipline: additive request keys only (rule 2), response shapes
untouched, pins changelog widened in-commit, locked by
TestOriginWriterAttribution (android lands, default stays web, unknown
normalizes). Full battery: only the two known pre-existing failures
(TestOPDSLibraryFolders rot, TestWebSocketPingPong — both reproduce
with the change stashed).
The stats page's Library-tab switcher (and every other native
<select>) rendered its dropdown LIST in the OS light scheme in dark
themes: the closed control is themed by its own classes, but the
popup list is UA-rendered and ignores page CSS unless the document
declares color-scheme (Chrome path) or the options carry explicit
colors (Firefox path). Both are now in place:
- :root { color-scheme: dark } (the default palette — Tokyo Night —
and all eleven themes except latte are dark),
- .theme-catppuccin-latte { color-scheme: light },
- select option { background-color: var(--bg-secondary); color:
var(--text-primary) } as the belt for engines where the popup
escapes the document cascade.
style.css regenerated via build:css:prod; verified served by the
rebuilt container.
VisibleLibrarySet becomes a package-level function taking the queries
(db-only services like statistics reach it without new wiring); the
LibraryService method stays as the ergonomic entry point. Statistics'
ids-shaper and the reader pages' three hand-rolled membership loops
(empty-rows fallback then linear scans) now delegate; the gates keep
their 403-on-invisible semantics (>null set membership, never silent
pass).
Census addendum from this pass: two OPDS sites call
GetUserVisibleLibraries directly but consume FULL ROWS (per-library
folder enumeration) — a third legitimate pattern alongside set/ids and
resolveLibrary's active-selection; documented, untouched. The library
LIST endpoint and its SSR data helper are row consumers too, by design.
collections.userVisibleLibraryIDs keeps its identity adapter and
delegates; the router's visibleLibraryIDsFor likewise (its identity
source is the session user_id string, the site-specific part it keeps).
The SSR site made the unification slog-proof: resolveLibrary — the SSR
one-of-a-kind (active selection + switcher rows + offline detection) —
stays untouched per the semantic diff.
Series' visibleLibrarySet was the third hand-rolled set builder (with
its own FromBytes-error idiom, the fourth idiom in four copies).
SeriesService.VisibleLibraries — a pure pass-through to
GetUserVisibleLibraries — now has zero callers and dies with the
migration.
One core for "which libraries can this user see": family-lens default-
allow stays where it always was (the GetUserVisibleLibraries SQL);
the core adds the shared set+ids shaping that four endpoints each
hand-rolled. Media's visibleLibrarySet is now a thin identity adapter
(JWT user object -> resolved id) and the download-permission gate uses
the set instead of a manual membership loop.
Semantic diff (H4.5 step 1, in the plan): zero family-lens divergence
found across all six sites — the rule lives in one query; the variation
was shape-level (identity adapter, set-vs-ids, UUID idiom).
The search feed carried a ~55-line inline copy of the acquisition-entry
skeleton, over a DIFFERENT sqlc row type (SearchMediaItemsRow vs
ListMediaItemsByLibraryRow) — the row-type split is why it stayed a
twin. The shared builder now takes opdsEntryInput, the narrow
ten-field subset the feed actually renders (deliberately not a row
converter), and acquisitionEntryOpts makes the two legitimate
differences explicit instead of incidental:
- labelLibraries: unscoped search labels entries with their library.
- includePdfAlternate: catalogs offer the PDF alternate for reflowable
books; search historically does not. Kept as-is — making search match
catalogs would be a behavior change, flagged in the opts doc.
The four OPDS behaviors tests are green in the test-runner container.
Flagged, not from this change: TestOPDSLibraryFolders fails identically
with the change stashed — pre-existing rot worth its own look (it never
ran in the hardening batteries because the -run patterns didn't match
it).
shouldEnablePanelDetection and ShouldTreatAsComic had identical truth
tables (manga/comics library x fixed_layout/comic_archive) under
different names — a drift trap: adding a library type to one silently
missed the other. One predicate now gates both features, with the
re-split instruction in its doc comment should the features ever
legitimately diverge.
The pending-hash-conflict walk existed twice (JSON admin API emitting
numeric counts, SSR page emitting a formatted summary) — deliberately
different shapes, but both duplicated the counts-or-zero lookup and
the five-field total. MediaItemUsageCountsTotal is the shared read;
the per-surface shaping stays local on purpose.
The H4 row-completion commit made abridged/audio_metadata emit on the
search wire (the endpoint serializes the row directly) — exactly what
the pin exists to surface. Whitelisted per the additive rule with the
changelog entry.
The unified search row lagged the table (survey overstated it —
narrators was already there). Both columns come from the
already-scanned media_items side of the joins: EXPLAIN before/after is
plan-identical (same Nested Loop shape, no new scans). Row-completeness
only — no client surface changes; the hand maps select their own keys.
SELECT mi.*, ci.excluded forced every caller to filter tombstone
exclusions in Go over a 69-field row, then a 45-field hand converter
(getCollectionItemsRowToMediaItems) copied it field-by-field into
MediaItems for BuildSections. The query now filters
(ci.excluded IS NOT TRUE) itself and returns sqlc.embed(mi) — callers
consume .MediaItems directly, the Go filter loops die in both
collection-detail ladders (JSON + SSR), and the converter is deleted.
Pins, H1/H3 tests, unit suites green.
checkRulesAgainstBook/evaluateRule and the service's EvaluateRules
existed in two parallel universes: one over database.MediaItems (what
every query actually returns) and one over ListMediaItemsRow (what two
hand-written 45-field mediaItemsToListMediaItemsRow converters
produced — verbatim twins in the handler and the dashboard service,
with a URL-resolution side effect baked into a "converter"). The rule
fields (genre/series/author/language/publisher/copyright_year/tags)
all exist on MediaItems, so the evaluators now take the real row and
the converters, their three call-site loops, and the dead
ApplyAutoAssignRules entry point (zero callers) are gone.
The rules surface is wire-covered by the collections rules scenarios;
handlers+services suites and the full pin/H1/H3 battery green.
ListConflicts, GetConflict, and the SSR media-detail active-conflict
lookup each hand-mapped their sqlc row (list-by-user, get-by-id,
list-by-media-item) into ConflictDetailResponse, triplicating the
optional resolution tail — including the silently-swallowed
ResolutionData unmarshal error, three times. NewConflictDetail is the
single mapper; a malformed resolution blob now logs instead of
vanishing.
Also extracts koreaderSyncEndpoints (the KOReader URL map duplicated
verbatim between token rotation and device approval; kobo's two URL
sets stay local because they deliberately differ).
Verified in the test-runner container (host conflict tests fail
environmentally on the container-only uploads path — baseline behavior).
The same pgtype unwrap-and-fill block (sync_enabled/auto_sync validity,
frequency fallback, pointer timestamps) existed at five DeviceInfo
construction sites, two of which also carried AuthToken.
deviceToInfo(device, includeAuthToken) replaces all five; token-bearing
lists pass the flag explicitly so the surface that exposes the token
stays a deliberate choice.
setOfflineMode/setOnlineMode only mean to flip sync_enabled, but they
called the full-row UpdateDevice with Go zero values — which wrote
DeviceName: "" and NULLed auto_sync/sync_frequency_minutes/
device_metadata on the device row at every offline->online flip. New
targeted SetDeviceSyncEnabled query touches exactly that column; the
real settings update keeps using UpdateDevice.
Caught as part of the H4 helper-census pass (plan §5.4 erasure list).
handlers textToString, router textToString, router getText, and utils
normText were four copies of the same NULL-to-empty unwrap (99 call
sites). One exported utils.PgTextString replaces them all; the private
normText (comparison normalizer) stays but is now the odd one out.
Harness fix found on the way: setupTestServer built a full wiring
literal for route registration and then RETURNED a second, sparser
literal that never copied AnnotationService — setup.AnnotationService
was always nil on the struct every test receives. Nobody noticed until
cross_library_move_test became the first caller (it previously used
the now-retired plain CreateMediaHighlight query); it now exercises
the real SaveHighlight path through the wired service.
H3 leftover: the annotation handlers route everything through the sync
service (dedup/LWW/tombstones), leaving the plain Create/Update/Delete
queries for media_notes/media_highlights/media_bookmarks with zero
callers repo-wide (handlers, services, tests). Queries.sql drops them
so H4's row work doesn't carry dead weight; sqlc regen compiles clean,
proving nothing referenced the generated stubs.
Also removes the literal duplicate `AND mi.missing_scan_count = 0`
line in the library-stats query (harmless but confusing).
Caught by the live round-trip sweep: a dedup-key re-save (a device echo
or a client re-POSTing a bookmark it already had) carried no locator
fields, and the full-column sync update wrote zeroes — which
pgFloat8/pgInt4 turn into NULLs — erasing the stored
percentage/page/cfi on every echo. The highlight stack had string
coalescing for exactly this reason (its locatorRefresher doctrine
comment); the numeric columns and the two cloned stacks never got it.
One primitive now carries the whole doctrine: coalesce[T comparable]
(zero value -> stored value), applied at the single write point of
each LWW apply — highlight (percentages, color, note_text,
chapter_reference added to its existing string coalescing), bookmark
(page/chapter/percentage/epubcfi/chapter_reference added to its
cfi/position pair), note (all locators). The redundant pre-coalescing
in the id-targeted resolution blocks is removed — the apply is the one
coalesce point, so dedup-path and id-path edits behave identically.
Verified live on the rebuilt container: dedup re-save and legacy
3-field PUTs preserve every stored locator on all three stacks; an
explicitly-sent field still lands. Pins, H1/H3 tests, unit suites
green.
The consolidation half of H3 — shared LOGIC with thin per-type remains:
- lwwDecision: the three compareIncoming* functions were the same
skeleton (client timestamp -> strictly-newer wins; no timestamp ->
any content difference wins) drifting apart only in per-type
equality and the schema-driven fallback timestamp (bookmarks carry
no updated_at). The decision now lives in one function; each compare
keeps only its equality expression.
- idTargetOwnership + tombstoneWins: the id-targeted resolution blocks
(HighlightID/BookmarkID/NoteID) shared the ownership check and the
deleted-row gate — now one gate each instead of three copies of the
resurrect semantics.
- queue.go: syncNote/syncHighlight/syncBookmark decoded device payloads
through three 10-key if-type-assertion chains; typed map accessors
(mapStr/mapF64/mapI32/mapDeviceSyncData) replace them — same
missing-key-is-zero semantics, declarative field lists. The progress
decoder is untouched (H1 territory, separately tested).
Per-type remains are the things that genuinely cannot merge: request
structs, dedup triples, the per-table sqlc param builders, and the
KOReader/Kobo payload adapters. Pins, H1/H3 tests, both unit suites
green.
H3 census bugs #2 and #3: both web PUTs used plain queries that
bypassed the sync-aware layer entirely — no LWW, no source bookkeeping,
and locator fields the client couldn't even send. The highlight PUT was
the only one already routed (via HighlightID); the other two now mirror
it:
- SaveBookmarkRequest gains BookmarkID, SaveNoteRequest gains NoteID:
id-targeted resolution reads the exact row, verifies ownership,
LWWs under its stored dedup key, and resurrects tombstones when the
edit is newer (same doctrine and comment structure as the highlight
path).
- Fields the PUT omits are coalesced from the stored row before the
full-column sync update — the legacy web PUT sends only
title/notes/position, and zeroing cfi/percentage/page on every edit
would have been data loss. Verified both directions by
TestH3_BookmarkPUTPreservesLocators (omitted survives, explicit
lands).
- The bookmark PUT's wire DTO gains the create endpoint's locator keys
as optional fields (documented additive request keys in
contract_pins.md); the note PUT keeps its content+position wire.
UpdateMediaBookmark/UpdateMediaNote plain queries are now uncalled
(the generated queries stay until the H4 query sweep).
H1's doctrine applied to the annotation service, which had been left on
setters: NewMediaHandler/NewKoboHandler/NewKOReaderHandler/
NewSyncQueueProcessor(WithConfig) now take it as a parameter; the four
SetAnnotationService setters are deleted; wiring updated (main.go,
router/sync.go, test harness).
With the service unskippable, the 18 nil-guards and their plain-query
fallbacks die:
- media.go: the create fallbacks for notes/highlights/bookmarks (the
bookmark one was the H3 census's Bug #1 — it dropped
percentage/chapter_reference the sync path carries), the highlight
PUT fallback, and the three delete-path plain deletes (tombstone is
now the only delete semantics).
- kobo.go: both markup-ingest annotation loops (the bookmark fallback
wrote the text into media_notes — a category error the fallback
never should have had).
- koreader.go: all three push-stack guards.
- queue.go: the three "annotation service not available" errors in
syncNote/syncHighlight/syncBookmark.
Behavior note: the deleted highlight create/PUT fallbacks were the only
paths passing note_id to the plain query — the sync service has no
NoteID field, so the web's highlight→note link was already being
dropped on the live path; this commit makes that pre-existing
limitation explicit rather than papering it over (surfacing note_id on
SaveHighlightRequest is left for a behavior commit that needs it).
Pins (now 20 incl. the 3 annotation CRUD suites) + H1 tests + both
unit suites green.
Pin-first batch for H3: freeze web annotation CRUD response shapes
(create 201 / update 200 / delete 204; full sqlc row models) before
the stack consolidation touches anything.
The new pins immediately caught a live server bug the 28 unit tests
could never see (annotations_test.go is pure-unit: keys and compare
logic, no DB): applyNoteLWW built UpdateMediaNoteForSyncParams without
UserID, so the query's WHERE id=$1 AND user_id=$13 compared against
zero — every dedup-hit note re-save (same content+position) returned
500 "update note: no rows in result set" and the update never landed.
The bookmark twin passes UserID and works; the note stack is the clone
that dropped the field.
Fix: pass UserID in the params (one line). Pins go green and become
the regression net; changelog records the new surfaces.
The last behavior gap in the H2 batch: has_conflict was part of the
frozen card shape but stamped false-forever on these three JSON
surfaces (only dashboard sections, search, and the SSR pages had it),
so every conflict-driven card action — the resolve-conflict CTA in
BookCard — never fired there. MarkActiveConflicts now runs after the
builder on GET /api/collections/:id, series books, and smart-collection
preview (all user-scoped; the endpoint already authenticates).
Key set is unchanged — has_conflict was always emitted, this only
makes its value truthful. All 15 contract pins + H1 tests green; the
pin's bookInfoKeys freeze unaffected.
All five branch shapes of GET /collections/:id now build cards via the
cards.go adapters: smart (BookCardsFromMediaItems), system-query (same
— gains format_group/badge it previously dropped), scoped
(BookCardsFromCollectionDashRows with the Excluded filter kept), and
the all-libraries branch whose rows distinguish ID vs MediaItemID
(BookCardsFromCollectionItemRows). LibraryTypeMap hoisted to once per
page; both []BookInfo{} error seeds die with the adapters' non-nil
empties.
The ExecuteSearch-driven bookshelf page fed SearchMediaItemsUnifiedRow
— which HAS FormatGroup and LibraryTypeName — into a bare 4-field
literal that dropped both, so bookshelf cards lost the media-type
badge the shelf once had. BookCardsFromUnified emits badge/format on
this surface (card-marking rides on; the SQL column strategy means
zero extra queries). audiobook shelves regain the corner glyph.
Both BrowseDetail surfaces hand-rolled bare 4-field literals (badges
dropped) and patched the nil-slice with []BookInfo{} seeds after.
BookCardsFromMediaItems feeds GetSeriesBooks/GetBooksByTag rows —
cards gain format_group + badge on the series and tag pages, and the
two seed blocks die with the adapter's non-nil empties. Conflict
stamping at these sites was already adjacent and is preserved.
router/search.go (the templ BooksGrid fragment surface) now uses
BookCardsFromUnified — the unified row's LibraryTypeName column gives
the badge with no LibraryTypeMap query; also drops that site's
needless pgtype.UUID byte-roundtrip on the cover URL.
The dashboard section builder — feeding both the dashboard JSON pin
surface and the SSR dashboard — now calls BookCardsFromMediaItems on
ds.Items. libTypes already arrives as a BuildSections parameter
(once per request); the literal only duplicated the badge lookup.
GET /api/series/detail?name=… books array via BookCardsFromMediaItems.
Series had the third LibraryTypeMap call INSIDE its per-item loop —
ListLibraries ran once per book on the page; now once per request.
POST preview-feed had the ladder's 14th hand-rolled literal. Now
BookCardsFromMediaItems + one LibraryTypeMap per request; the response
carries format_group/badge values for the first time (all seven keys
were already emitted as zero-value JSON, so this is a value change on
a pin-frozen shape, not a key change).
GET /api/collections/:id books — all four branches now build cards via
the cards.go adapters. LibraryTypeMap hoisted to once per handler (it
was 1 query per item in the smart and no-library branches). The system
branch — which before stamped cards BARE (no format_group, no badge —
the ladder's only JSON gap) — now emits the same card shape as its
siblings by feeding section.Items through BookCardsFromMediaItems,
matching what BuildSections produces for the identical data. The
collection-error seed dies via the adapters' non-nil empty slices;
the no-matching-section nil result is preserved as-is (that branch
never seeded — today's 'items': null stays today's 'items': null).
Pins: CollectionsJSON, DashboardSections, MediaListBothBranches green.
BookInfo, LibraryTypeMap, and MediaBadgeKind move verbatim out of
collections.go into handlers/cards.go (the card package surface, same
package: no import-graph or templ changes), joined by the H2 builder:
- buildBookCard (unexported): the ONLY place BookInfo is constructed,
fed by cardSource; cover URL via ResolveMediaURL, badge via
MediaBadgeKind — both inside the builder, not per call site.
- Four exported adapters normalize the four real feed shapes:
database.MediaItems (sections/smart/series/tags/preview),
GetCollectionItemsForDashboardRow (collection dashboard branch),
GetCollectionItemsRow (collection-items joins — MediaItemID field),
SearchMediaItemsUnifiedRow (badge rides its LibraryTypeName column,
so no LibraryTypeMap query at all).
- Adapters return non-nil empty slices, which retires the per-site
[]BookInfo{} empty-seed assignments when call sites migrate.
No call site changes in this commit — the 20 literals and 5 seeds are
migrated one surface per commit next, each with the H0 pin suite.
Verified: build, vet (only the two pre-existing flagged saveCoverImage
warnings), and 14/14 contract pins — the key freeze is byte-identical.
2026-10-17 lessons from the hardening session: most test/bruno failures
were harness/schema/environment drift (drifted body keys, truncated
payloads, container-only paths), found by triaging the test before the
server. Codified as: triage order (test schema -> environment -> cheap
probe), with the guard against "writing tests to work" — test/contract
edits are asymmetric with code fixes: only harness plumbing fixes,
additive pin widenings with a changelog entry, or deletions of tests of
removed behavior. Pins were green at a known commit; git history
arbitrates stale-vs-changed. A right-looking test that fails is prima
facie a server bug: stop and flag.
contract_pins.md rule 4: narrowing/removing a pinned key requires the
user's explicit sign-off — a pin freezing buggy behavior is a ledger,
not a license.
Book-side twin of the audio-survival test, closing H1's verification
list. Scenario: web push carries percentage + epubcfi + context_text;
a koreader-style percentage-only push from a second source within the
window detects a conflict; the user resolves with the CFI-carrying
side. The funneled resolution must land the winner's ENTIRE locator
pair — percentage 0.5, epubcfi, context_text together (the old direct
writer NULLed context_text and character_offset, breaking the coherent
pair the far-anchor work established) — and must not spawn a duplicate
pending conflict.
Follow-up to the sweep route audit, correcting its diagnosis: the CLI
DOES read .yml bodies — via OpenCollection's `body.data` string form
(the NewDevDBSetup script always worked because it used it). The
failing scenario POSTs used `body.json` (structured) / `body.jsonBody`
(string) — keys the CLI silently ignores, sending Content-Length: 0
with form-urlencoded content-type, so every server bind failed.
- 81 bodies converted to data-form (json maps serialized; jsonBody
strings kept verbatim, {{vars}} intact).
- 6 pre-existing TRUNCATED payloads repaired: the original generator
cut bodies at the first {{var}} (every SHA-256-identified scenario
plus Sync Progress/Sync Bookmarks shipped mid-JSON and 400'd from
day one). Rewritten whole, faithful to each file's documented
purpose; book_sha256/book_uuid added to the environment.
- environments: opds_base_url correction from the sweep retained.
Verified live against the rebuilt dev container: the full koreader
scenario tree runs green through `bru run` — 14/14 (202s across
immediate/checkpoint/SHA-256 progress, 200s bookmarks, highlights,
notes; SHA-256 book identification exercised against a real
file_sha256).
Audited every request (247 .yml files) against the routes the server
actually registers (extracted from internal/router, group prefixes
resolved). Results:
- 44 stale requests: deleted where the endpoint no longer exists
(universal-progress GET/PUT /api/progress/:id — routes never existed,
the handlers were deleted in H1; queue stats/clear-failed/process;
admin profile page; auth/account) and FIXED where the endpoint exists
under a different path (kobo token-in-path scheme: every
/api/sync/kobo/<seg> and /api/v1/kobo/<seg> URL now carries
{{kobo_device_token}}; /sync/* missing their /api prefix; koreader
/api/v1/koreader/sync/* -> /api/sync/koreader/*; link-book and
unlinked-books moved to /api/devices/:id/sync/*).
- koreader Sync Highlights/Sync Notes pointed at endpoints that never
existed with a body shape to match — rewritten against POST
/api/sync/koreader/bookmarks (the annotations funnel carries
highlights and notes arrays).
- 9 files the GUI refused to load: 4 had bruno-invalid script type
"tests" (converted to after-response), 4 were info/type:collection
doc files with bogus http blocks (converted to folder.yml folder
docs), 1 had a dead pre-YAML schema and no route (deleted).
- devices/koreader requests now carry request-level bearer auth
({{koreader_device_token}}) — they inherited the collection's user-JWT
bearer, which device endpoints rightly reject — and their bodies
reference {{media_item_id}} instead of dead placeholder uuids.
- environments: opds_base_url was empty (every OPDS request missed the
/opds prefix); set to http://localhost:8765/opds and un-doubled the
prefix in opds/scenarios.
Known limitation, for the handoff: bru CLI cannot execute .yml request
bodies (the collection is GUI OpenCollection format; the CLI runner
reads bodies from .bru files only). A .bru migration is the remaining
bruno work if CLI runs are wanted.
Found while verifying H1's "app offline-play -> flush -> position lands"
with the emulator path: the flush has been dead code in two independent
ways, invisible because the first break made the second unreachable.
1. ListPendingSyncQueueItems matched `device_id = $1` with a NULL $1
(the processor passes DeviceID{Valid:false} to enumerate all devices;
uuid = NULL matches no rows). The ticker therefore processed nothing,
ever. The WHERE now treats NULL as "every device" (the same
narg::uuid IS NULL idiom the visibility queries use; EXPLAIN'd — seq
scan on a bounded table, idx_sync_queue_status_priority_created
fits). The processor also dropped its fake-pagination loop, which
re-fetched the same first batch forever once the backlog hit the
batch size (no OFFSET in the query).
2. dbItemToSyncQueueItem dropped SyncData — even with a working
enumerator, every flushed item arrived with an empty payload and
failed JSON unmarshal. Dead code harboring silent breakage, exactly
the plan's thesis.
Callers updated for the renamed RowLimit param (queue handler, offline
detector). TestH1_SyncQueueFlush_LandsAndPreservesAudio drives the real
flow: web PUT seeds audio 900s, a pending checkpoint-style queue row
applies percentage 0.8 through SaveProgress, audio survives, queue row
completes.
Also fixes dashboard_test.go's stale BuildSections signature (the
package's test binary has been unbuildable) — user-approved: tests
designed against old shapes are not worth keeping.
The cleanup in setupSyncTestDB ran DELETE FROM sync_queue WHERE true —
a table-wide wipe that discards REAL devices' pending offline entries
on every suite run. Same class as the 2026-10-14 reading_history
incident the standing rules ban. The test's own queue items cascade
from its 'Test %'-named device, so the wipe is redundant for cleanup —
scoped it to those devices (defense in depth ahead of the cascade).
Flagged during H1 verification: the line executed during an in-
container suite run this session; any device with pending offline
queue entries at that moment should re-push on its next sync.
Batch H1 step 4:
- Handler GetUniversalProgress (progress.go) deleted — no route mounts
it; only /api/progress/:id/history (GetProgressHistory) and the SSR
/progress page exist. Note: the bruno universal-progress suite
documents GET/PUT /api/progress/:id + /api/progress/device/:device_id
— those routes do NOT exist server-side; that suite is stale
documentation, flagged in the plan.
- Handler GetAllProgress deleted — unregistered near-copy of
GetAllProgressData (which the /progress SSR page uses and stays).
- series.go: dropped the `_ = libTypes` no-op (libTypes IS used by the
Badge line right below — the assignment stays).
- router/helpers.go: visibleLibraryIDsFor's signature and first
statement were fused onto one line; split for readability. No
behavior change.
Batch H1 step 3b — THE data fix. Conflict resolution was the one live
erasure writer: applyProgressResolution and its verbatim twin
applyResolution both wrote the winner via a raw UpdateUniversalProgress
whose param list omits audio_offset_seconds/context_text/viewports, and
the SQL's DO UPDATE SET ... = EXCLUDED... NULLed exactly those. The web
UI resolves through POST /api/conflicts/:id/resolve and
/api/conflicts/bulk-resolve (main.js resolveConflict/bulkResolve), and
the reader's mark-as-read fires a resolution first when the reader SSR
stamps data-conflict-id/-winner — so every resolution today erased the
audiobook position and reader context.
Now: ProgressService is a required ConflictHandler dependency; BOTH
call sites (single resolve + bulk resolve) share one
applyProgressResolution that funnels through SaveProgress (the verbatim
twin is deleted). Deliberate choices, per the plan:
- SkipConflictDetection on the resolution write: both flows apply
BEFORE the conflict row is marked resolved, so the funnel's LWW
detector would otherwise file a duplicate pending conflict
(HasRecentConflictResolution is false at apply time).
- Source "conflict_resolution" — the row's last-write attribution; the
winner stays durably recorded in resolution_data.winner.
- Broadcast: true — live devices jump to the winning position.
- The winner snapshot maps percentage/epubcfi/context_text/chapter/
character/page/total_pages; a winner without a CFI clears the locator
pair (ClearLocator) instead of re-pairing with the loser's stale
anchor. Everything the snapshot doesn't carry SURVIVES via the LWW
seed: audio_offset_seconds above all. chapter_progress keeps its
stored value (was overwritten with the book percentage), an absent
percentage survives (was written as 0%).
- Accepted: the resolution captures one reading_history session
(attributed to conflict_resolution; the 2h idle ceiling applies).
Verification: TestH1_ConflictResolutionPreservesAudio drives the real
flow — web push with audio 300s, diverging push from a second source
detects a conflict, resolve with the second source as winner, GET shows
percentage 0.3 AND audio_offset_seconds 300 (previously NULLed), and no
duplicate pending conflict remains. Full contract pin suite green.
The host-run TestConflict*/TestProgress* failures are environmental
(container-only /app/uploads path + db hostname; fail identically at
baseline b4ae423) — the same suites pass in the test-runner container.
Batch H1 step 3a. The ceiling census (exhaustive UpdateUniversalProgress
grep) found 9 nil-guarded `if progressSvc != nil { funnel } else { raw
erase-write }` fallbacks — the survey's "four" under-counted kobo.go,
which has FOUR sites (Markup ReadingSync, Markup last-read-place,
AnalyticsGettests, SyncFromServer), plus one each in media.go, queue.go
and koreader.go. All unreachable: the service is created unconditionally
at main.go:127 and wired into every handler and the queue processor; the
test harness wires them too.
The erasure pattern is now unrepresentable:
- NewMediaHandler / NewKOReaderHandler / NewKoboHandler / both
NewSyncQueueProcessor flavors take the ProgressService as a REQUIRED
constructor arg; the SetProgressService setters are gone.
- Every fallback body (raw UpdateUniversalProgress with explicit-NULL
pgtypes for viewport/scroll/panel/reading_mode/zoom and no
audio_offset_seconds/context_text) is deleted; each write funnels
through SaveProgress with the LWW seed, so fields the writer doesn't
know survive.
- Removed the now-dead koreader pg-type helpers (textPtrToPgText,
intPtrToPgInt4, int64PtrToPgInt8) and their self-tests.
Behavior deltas (stored-row content, no wire shape changes —
KOReader/Kobo response pins stayed green):
- KOReader fallback used to write explicit NULL viewports and drop the
context_text the koplugin sends; the funneled mainline already maps
book.ContextText.
- Queue flushes no longer overwrite chapter_progress with the book
percentage (bogus data; preserved via the seed).
- A web PUT without percentage no longer writes 0% via the deleted
fallback; percentage seeds from the stored row.
Contract suite: KOReader×4 + Kobo×3 pins green.
Batch H1 steps 1-2 of the hardening plan. Wire contracts frozen; the
pinned shapes are unchanged modulo the whitelisted additive GET keys
(docs/contract_pins.md).
1. One canonical request struct. The web PUT binds SaveProgressRequest
directly (identity/attribution fields are json:"-" so they stay
server-owned) and copies its server fields once — the hand-rolled DTO
and the 20-line field-copy block are gone. Parse set for existing
clients is IDENTICAL to the old DTO (verified in web/src: the ebook
reader sends percentage/current_page/total_pages/reading_mode/epubcfi/
context_text/chapter/zoom_level — reader.ts:3113-3143; the audiobook
reader sends percentage/chapter/audio_offset_seconds/elapsed_seconds/
reading_mode — audio.ts:518-520).
2. GET /api/media-items/:id/progress now serializes through one
row-struct-driven builder instead of a hand-map, so a row column
cannot be silently forgotten again (the audio_offset_seconds
omission). Existing keys are verbatim; the row already carried
audio_offset_seconds/reading_mode/viewport_x-y/zoom_level/
scroll_position_x-y/panel_number and they surface as WHITELISTED
additive keys. The web audiobook reader already reads
audio_offset_seconds (audio.ts:219) — with the key absent,
durationless books resumed at chapter start (offset always 0); they
now resume correctly. The conditional context_text bolt-on and the
server-derived css_selector/anchor_href block are unchanged —
restore precedence stays percentage-first, never re-derived.
Batch H0 of the hardening plan: freeze the JSON key SETS of every
load-bearing surface so H1-H7 can only change them additively.
- cmd/server/tests/contract_pins_test.go: 14 tests / ~30 subtests pinning
progress GET+PUT, the KOReader plugin surfaces (progress push/pull,
library, bookmarks), the Kobo adapter shapes (initialization, markup,
sync-from-server), collections detail all four branches + preview,
series JSON, search rows, dashboard sections, media list BOTH branches
(raw sqlc rows vs resolved 38-key map), and SSR smoke for
bookshelf/collections/series/tags.
- docs/contract_pins.md: the pinned-shape inventory + the rule that a
batch widening a shape must widen the pin and this doc in one commit.
Suite is green on the unmodified tree at b4ae423. KOReader/Kobo/progress
pins are the hard freeze (the plugin is an external client).
The daily charts' x-axis labels were bare MM-DD ('10-15', '10-16') — on
the Listening per Day chart (y in hours) those read as hour values, which
is exactly the '10-16 hours a day' misread that prompted this. Labels now
spell the month ('Oct 15') and every chart's tooltip carries its unit
(min / pages / hrs). main.js?v=5
The /stats page gains a second tab (Reading | Library) implementing the
Audiobookshelf-style library dashboard (HANDOFF item 9): media_items
INVENTORY aggregates — items, hours/pages, distinct authors, size,
distinct narrators, Top Genres / Top Authors bars, Longest / Largest
items — for one library or the All-libraries aggregate.
- GET /api/stats/library-usage?library_id=<uuid|omit>; family lens
enforced: a library_id outside the requester's visible set is a 403,
never a silent pass (AGENTS.md standing rule; 431fec0 history)
- six sqlc queries with the visibility pattern + healthy-item filter
(archived_at IS NULL AND missing_scan_count = 0); hours_mode lets the
client swap the Hours/Pages card and the Longest ranking per scope
- Reading tab unit audit (HANDOFF item 3, settled 2026-10-16): minutes
are the merged currency everywhere honest; only the PAGES unit splits
— Pages card goes mode-aware (Pages · text / Listening Time in an
audio-only pool) and the charts row gains Listening per Day (hrs)
when audio minutes exist in the range; summary daily buckets gain
audio_minutes split by the item's library type
- tab state in the URL (?tab=library&library_id=…); selector fed by the
existing /api/libraries/visibility endpoint; main.js?v=4
- integration tests: family-lens 403 pin, hidden-library exclusion
(differential), seeded text+audio aggregate correctness, auth gate
- docs/developer/api/stats/get_library_usage.md
Known-remaining suite failures are environmental: TestConflictList_Empty
+ TestWebSocketPingPong collide with the concurrently-running app session
(live emulator pushes + scans on the shared dev DB), and
TestAdminCreateUser's timezone expectation drifted from live system
settings — all three fail on the clean tree too.
A2 made narrators/narrators_search NOT NULL DEFAULT '{}' (86f0af6), but an
explicit NULL in an INSERT/UPDATE column list bypasses the column default
— the API media-create path (POST /api/media-items without narrators)
500'd on every call, taking the whole integration suite down with it
(ISBN tests, capture seeds, all direct-DB seeds).
- set_library_type_name() trigger now coalesces NULL narrators/
narrators_search to '{}' at the row level, and its update trigger
widens from BEFORE UPDATE OF library_id to BEFORE UPDATE so every
write path is covered
- defense in depth at the callers: handler create/update normalize nil
to empty slices (normalizeNarrators), scanner seeds the same
(nilToEmpty) for files that carry no narrator credits
- analytics device-usage tests move to the dedicated per-test user
(the shared admin's device list accrues live rows mid-suite — the
documented testuser flapping class)
Reader fullscreen: the topbar gains a fullscreen toggle (icon state + f
shortcut + help row; f is guarded so Ctrl/Cmd+F stays find-in-page).
Enter requests the Fullscreen API with fallbacks (desktop/Android get a
true full display; iOS Safari has no page-fullscreen API and falls back
to the soft visual mode only). Margins: the foliate paginator's closed
shadow root only yields through its observed 'margin' attribute, so
fullscreen fits a comfortable ratio (6% of the short side, floored at
48px) and floors it by the measured env insets — Android reports true
cutout insets in fullscreen, so portrait phones keep text clear of the
camera band; iOS letterboxes are already env-padded chrome. Fixed-layout
renderers skip it (fxl/webtoon pack pages edge-to-edge by design).
Exits the user makes outside the toggle (Esc, browser UI) sync via
fullscreenchange(+webkit) and restore the ordinary 48px margin; a
position-keeping re-render follows the ResizeObserver re-entry path.
Verified on-device: enter -> true fullscreen + icon swap + camera band
clear, exit -> chrome back, margins restored, position kept.
Volume: revert the stacked panel — the popup returns to the liked shape
(mute button left, horizontal dragger right, dialogue above the strip);
one-line flex restore of the pre-89665fb layout.
Manga badges: MediaBadgeKind makes the library type authoritative for
the comic/manga split — manga volumes ship as fixed_layout scans as
often as comic_archive (and manga_type is unreliable across sources:
unknown/NULL on real rows), so the old comic_archive+manga mapping left
the scan volumes unbadged. Anything in a manga library reads manga;
anything in a comics library reads comic. Verified: /dashboard lands 24
title="Manga" cover badges for the Manga library and the sections API
reports badge=manga for a fixed_layout item.
Ghost sidebar: the >=1024px media block applied body
padding-left:var(--sidebar-w) unconditionally, so pages that never
render the sidebar (logged-out pages, the audio player, the reader
itself) carried a phantom 16rem left margin that pushed content off
center on desktop widths. The body:has(.app-sidebar) rule below was
already the correct conditional source — it is now the only one.
Verified in the built stylesheet: exactly one such rule remains.
Infra: main.js ?v=3 stamp bump (reader.ts changed; hashed chunks churn
their URLs themselves).
Annotations banner: audio/text bookmarks join notes and highlights —
third row kind (bookmark glyph indicator, H:MM:SS / Page N / Chapter N
friendly labels), edit-note/delete actions branch by kind, and the
banner's export now covers all annotations. The archived ledger
(ListDeletedAnnotationsForBook) already tombstoned bookmarks equally
and Restore/Purge take annotation_type=bookmark — verified live:
delete → restore brings the row back with its note intact.
Volume: the dragger sits BELOW the mute icon in the panel (stacked
flyout shape) with the horizontal slider from the last round.
Badges: the real fix for comics/manga — format_group only knows
'comic_archive' (manga_type is unreliable across sources), so the
comic/manga split is resolved from the LIBRARY type server-side.
handlers.MediaBadgeKind + BookInfo.Badge at every construction site
(dashboard SSR + sections API, collections, series, and the bookshelf
search grid — which was also missing FormatGroup entirely, so no card
ever badged there). Both renderers (templ + dashboard.ts) switch on
the resolved badge. Verified: All-Books grid renders 2 Comic + 3
Manga badges, dashboard the audiobook one.
Infra: header.templ main.js src gains ?v=<stamp> — browsers keyed an
intermediate bundle by URL and served it across rebuilds (the stale
'Add note' the user saw); bump the stamp when JS stale-shadows.
Bookmarks: notes are markdown now — capture and drawer edit are
textareas (raw source), display renders through the reader's
markdown-it + DOMPurify pipeline (renderNoteHtml) as x-html; the
edit textarea drops the enter-to-save handler so newlines survive.
Volume: the panel dragger is horizontal (mute button beside it,
7.5rem slider) per user feedback — vertical orientation read as a
scrubber clone at a glance.
Play/pause: unicode glyphs replaced with SVG — the triangle is
optically centered (2px right-bias) and the pause bars are longer
and thicker rounded rects (4.2×15 at rx 1.7).
Dashboard: the client-side card renderer (dashboard.ts — the path the
JSON sections actually render through) now mirrors the templ BookCard:
cover-corner badges for audiobook/comic/manga (books unbadged) and the
hover action shows headphones with 'Listen to …' labels for audiobooks.
BookInfo type + Go struct carry format_group end to end.
Verified on-device: pause glyph centered/thicker, horizontal volume
panel, markdown note ('**bold**' + list) rendered in the bookmarks
drawer, badges on dashboard cards, sections JSON carrying
format_group.
Transport row is now the five-button audiobook shape the user asked
for: previous-chapter · circular-skip-back (seconds displayed inside
the ring) · play/pause · circular-skip-forward · next-chapter — the
ring arrows are stroke SVGs (rotate-ccw/rotate-cw) with the number as
an overlay so it stays upright and follows the skip setting.
Settings strip collapses to four controls: Skip dropdown, Speed
dropdown, Bookmark (capture popover unchanged), Volume. Volume opens
a panel with a vertical dragger (writing-mode:rtl slider) and a
mute/unmute button; the strip icon is dynamic — 1–3 arcs by level (low
<34%, mid <67%, high) and crossed when muted or zero. Volume/mute
persist in localStorage and restore with the element; unmute floors
zero-volume to 50% so unmuting is audible.
Verified on-device: dropdowns open upward with current value
highlighted, mute flips both icons to the crossed state, slider
drag changes arc count, chapter buttons jump (pill → 15/34) and the
caption tracks the drawer's chapter labels.
A2a (metadata round): narrators TEXT[] + narrators_search (tags
pattern: accumulator + GIN pairs) + abridged bool on media_items;
scanner fills from the Libation/ABS sidecar (narrators verbatim,
abridged flag) with audio-payload promotion; editorial fields ride
the real columns while audio_metadata stays scan-derived (duration/
codec/chapters). Detail/edit/save paths carry the fields; the web
edit dialog grows an Audio section (Narrator(s) comma-input, Abridged
toggle, scanned duration read-only); search weights + narrator filter
(word_similarity over narrators_search) + narrator autocomplete added.
A2c (per-media-type wording): copy helpers keyed on format_group —
Listen Now, Mark Complete/Not Complete, Listening Progress, Your
Listening Stats, Time Listening, Time listened (pages tile swapped for
audio), Position H:MM:SS / total H:MM:SS on detail + progress pages
(duration parsed from audio_metadata). Dashboard: system collections
swap to Continue Listening / Recently Listened at DISPLAY time when
the audiobooks library is selected (stored names unchanged; restores
keep working); week card = Listening Time / Sessions / Books Touched;
cover-corner badges identify audiobook (headphones) / comic / manga
cards on mixed grids — books stay unbadged; hover action icon follows
the media type.
A2b (player): the scrubber binds value to the playhead with an explicit
scrub guard (the handle previously froze at its initial max); header
row = reader-style back arrow + Chapter n/total launcher; chapters
open in a floating top-down drawer (current chapter highlighted, page
never shifts); Skip/Speed rows share one label-left layout; honest
wall-clock session accounting (playing-only deltas, paused flush).
A2d (bookmarks): note-bearing time bookmarks ride the existing
media_bookmarks substrate (notes column + sync-aware service already
there) — audio locator convention audio:<seconds>, chapter_number
filled, auto-title '<chapter> — H:MM:SS', capture popover with optional
note, drawer with jump/edit-note/delete. Zero server changes needed.
Verified live on the Libation sample: narrators backfilled by rescan;
PUT edit persisted + overrides detected (narrators, abridged); force
rescan preserved the user edit; reset-to-scanned-of-defaults restored
truth; narrator filter/autocomplete scoped by library; dashboard
listsening wording + badges render; bookmark save with note deduped to
one row.
Phase 1 (foundation): 'audiobooks' library type seeded (schema + DB);
audio_metadata JSONB on media_items (the kobo_metadata precedent —
only audio paths read it); reading_progress.audio_offset_seconds for
the honest {chapter, offset} locator; extensions/mimes registered for
ALL breadth formats (m4b/m4a/mp3/flac — deliberate per the DOC/LIT
doctrine, m4b scans richly first); classifyFormatGroup 'audiobook'
case (gotcha 39 recomputes on rescan); Go mime table extended at boot
so media elements accept the streams; ffmpeg/ffprobe in the image
(8.1.2 — also the xHE-AAC identification tripwire).
Phase 2 (ingest): sidecar-first (Libation/ABS metadata.json —
narrators/ASIN/curated chapters), ffprobe fallback/filler (duration,
bitrate, codec/profile, embedded chapters). Fixed three real
parser brittlenesses the Libation sidecar exposed: series accepts
string OR array-of-objects, publishedYear accepts number OR string,
ffprobe chapter timings accept number OR string (flexFloat). Folder-
structure fallback now runs on the UPDATE path too (rescans were
clearing sidecar-absent series/author); audio facts re-derive on
every rescan (gotcha 39).
Phase 3 (playback + sync): GET /api/media-items/:id/chapters (the
format-abstraction list); streaming rides the existing cookie-authed
Range /uploads path; PUT progress carries audio_offset_seconds +
elapsed_seconds (honest wall-clock playing time) — the reading-
history capture prefers player-reported elapsed over the inter-push
delta, same 2h ceiling; reading_mode='audio' marks the row.
Phase 4 (web player): dedicated audio reader page (foliate not
involved) — cover, chapter title, whole-book scrubber, skip ± (chips
1/5/10/15/30s, default 10s), speed (0.75×–2×), chapter list with
jump-to, Media Session lockscreen/notification transport, periodic
playing-only progress pushes, position restore on open.
Verified live: ingest of the Libation sample (Gardens of the Moon —
title/author/series/cover/date from sidecar+folders, 34 chapters,
26h duration, aac/LC); chapters endpoint; the reader route serving
the player; playback with position restore (0:57 round-trip);
progress rows in audio mode; wall-clock session rows in
reading_history.
Completes the reader-chrome UX batch (handoff item 7): the gitea
mirror now carries fork 0.5.4, so the pin replaces the temporary
node_modules patch the verification build used. Rebuilt and re-verified
live (chrome starts hidden; flick-down reveals).
The chrome previously woke on ANY activity: pointermove (mouse drift
and, worse, the touch drag of a page turn itself), every keypress, and
every touch tap toggled it. Now reveal requires explicit UI intent and
reading actions suppress:
- Desktop: cursor into the top 10% / bottom 8% band raises the chrome
(screenY-based so content iframes classify identically); content
pointer movement never does. Mouse center-clicks no longer toggle.
- Touch: a vertical flick (dominant + fast) reveals; while visible,
flick down OR up dismisses. Plain taps are fully inert — a stray tap
can never flash the UI.
- Keyboard: page keys never poke; turning (keys, mouse edge zones, or
a touch swipe via the snap-relocate) actively dismisses visible
chrome.
- Highlight saves return to a clean screen: the selection flow owns
the screen and the post-save pointer/keyboard pokes are gone.
- Chrome starts hidden; always-visible pin and auto-hide timer remain
(bands use a shorter 1.2s fade).
Pairs with the foliate-js 0.5.4 paginator change (scrolled layouts
scroll manually so the flick gesture is observable in webtoon); the
fork pin bump follows once the gitea mirror shows f622841.
Extends the device-authed koreader surface with the handoff item 4
Batch B contract:
- GET/POST /api/sync/koreader/rating — the device mirrors the .sdr
summary's 5-star rating against the 1-10 server scale (POST 0 clears).
- GET/POST /api/sync/koreader/review — raw-markdown review mirror
(summary.note); empty body clears, matching the first-party PUT.
- POST /api/sync/koreader/stats — the device pushes its CUMULATIVE
total reading seconds; the server watermark (highest total ever
converted, read from the koreader sessions' own device_metadata)
converts only the delta into a reading_history session attributed
to the koreader device. Reinstalls cannot double-count; sub-minute
deltas ride forward (no session, no watermark movement); a lagging
device clamps to zero instead of dragging the total back.
Book identity resolves through the existing priority chain (uuid →
sha256 → alias → title/author). GET binding uses query tags; POST
endpoints bind the ref+payload once (an earlier resolve-then-bind
draft drained the body).
Live-verified against the dev server with the koreader device token:
rating set/get/clear, review mirror, and the stats sequence
900→session, 960→delta session, 970→rides forward, 500→clamps; the
book's stats card and the By Device panel both pick up the split.
media_reviews table mirrors media_ratings' shape (unique per user+item,
cascade deletes, updated_at for LWW) but standalone: a review without a
rating must work. Body is RAW MARKDOWN — storage and every payload carry
the source; rendering happens only at display sites (same doctrine as
notes, same sanitization pipeline: markdown-it + DOMPurify).
API: GET/PUT/DELETE /api/media-items/:id/review (PUT upserts; an empty
body deletes — clearing via an editor is a delete, not an empty row).
Book-detail page payload now carries the review beside the rating.
Page: the review renders above the synopsis as a scrolling card (same
max-height box as the summary), hydrated client-side through
renderNoteHtml; 'Write review'/'Edit review' sits next to the rating
stars and opens a raw-markdown editor dialog (save / delete / cancel).
sqlc regenerated; schema applied to the dev DB. Live-verified: upsert,
get, empty-delete, page markup, and the app round-trip against the dev
server.
Bug: the admin 'Scanning Libraries' widget fed the server's 0-1
progress straight into percent strings and bar widths, so the overall
and per-library percentages capped at '1%' and the bars never filled.
Both now scale by 100 (matching the setup page, which was already
correct).
Bug: scan state only reached clients via websocket events, so a page
opened (or refreshed) mid-scan showed nothing until the next tick —
and the admin widget never re-attached at all. The worker now exposes
active jobs (with their library_id so rows keep their names), served
as GET /api/scanner/active (admin, re-attaches the widget + polling)
and GET /api/scanner/active-summary (any user, restores the sidebar
indicator on load). Idle hides both, exactly as before.
Annoyance: the sidebar's 'Scanning… N%' was a block element between
the logo and the nav, shoving every menu entry down while a scan ran.
The percent now lives inside the fixed-height logo row next to the
spinner (spinner last, so growing digits extend leftward); nothing
below it moves.
The htmx self-refresh was bound to an element that was itself being
destroyed: the widget rendered inside #collections-container, and the
switcher's renderDashboardCollections replaces that container's whole
innerHTML on every library switch — listener and all — before any
refresh could fire. Hence 'widget disappears instead of updating'.
The widget now renders in its own strip between the switcher subbar
and the sections container, which the switcher never touches. The
week-refresh htmx re-fetch (with the current #library-select value)
then has a living target and re-scopes the numbers per library.
Verified: the subbar's refresh button drives the identical
re-render path and the widget persists with its data.
sync_integration_test's cleanup ran DELETE FROM reading_history WHERE
true — harmless for years while the table sat empty, and a silent
stats-data destroyer the moment session capture started feeding it:
every suite run erased the shared dev accounts' captured reading
(the user's manual flip-through test data included).
The cleanup is now scoped: the sync tests' sessions belong to their
test-sync users, and deleting those users cascades their rows. Any
future table-wide cleanup DELETE is off the table (AGENTS.md test
rules updated).
Three dashboard complaints, one build:
- The widget collapses to a single summary row (This Week · minutes ·
pages · sessions · books · streak · Full stats) on every screen size
and expands to the full grid via a details/summary toggle — the
carousel sections start immediately below it again.
- The 'Your Library' heading moved into the library switcher subbar
(LibrarySwitcher gains an optional title param; collections/series
keep the plain 'Library' label). The standalone heading block is
gone.
- Root-caused the JS-switch vanishing: renderDashboardCollections
replaces ALL of #collections-container's innerHTML, so anything
server-rendered inside it (title, widget) died on the first library
switch. The widget now re-fetches ITSELF: a /dashboard/week-widget
fragment (same resolveLibrary family-lens rules as the page) is
pulled by htmx whenever the switcher dispatches week-refresh —
carrying the current #library-select value — so the widget both
survives switches and RE-SCOPES to the switched library. Initial SSR
renders the same scope the sections use.
ThisWeek (and the streak calendar) gained an optional library filter.
Tests: TestReadingStatsWeekLibraryScope — per-library totals, scoped
streaks, invisible-library isolation, malformed id 400. Full suite
PASS.
The bucket name read as confusing — 'unsynced' implied data loss when
the push is always captured; it only lacks a device credential. The
bucket is just 'Web' now (service fallback, stats.ts fallback, panel
copy, tests).
Also: analytics_test exact-zero assertions (EmptyHistory,
NoReadingHistory) ran against testuser, which is a LIVE reader — real
sessions accrued on it mid-suite and both flapped. They now run under
the fresh regular user, like the stats tests; the 'testuser is a live
reader' rule is recorded in AGENTS.md.
- Attribution: registered clients may now send their device token in
X-Device-Token on PUT /api/media-items/:id/progress (the user JWT
stays primary). The token must map to a device OWNED BY the caller —
foreign/invalid/absent tokens soft-fail to plain web attribution so a
stats decoration can never break a sync. Attributed pushes capture
device_id (FK-safe), device_metadata.source = the registered device
name ('Pixel 9', 'Kindle Paperwhite'), and last_sync surfaces show
that device. Side effect worth having: app-vs-web pushes used to be
LWW-invisible (both sources read 'web'); they now surface as
legitimate cross-source progress conflicts when they diverge.
- /stats gains a By Device panel alongside By Library (grid 3-wide):
reading minutes + sessions + books per registered device, plus one
Web / unsynced bucket for device-less pushes so the panel always
reconciles with the headline totals. Range-aware, family-lens
scoped; renderBars shared between panels in stats.ts.
- The device identity IS the registered platform: devices rows already
carry the name/type the client registered (app sends Build.MODEL +
'mobile' at /api/devices/register, KOReader/Kobo likewise) — no new
identity plumbing on the server.
Tests: capture attributed to the device (row device_id + source
metadata + last_sync_source), foreign/nonexistent tokens soft-fail,
device buckets reconcile (80 attributed + 75 unsynced = 155 total).
Full suite PASS.
The design parked in the handoff (multi-tier stats off reading_history;
bookorbit-style in data, not looks) is in. The real prerequisite work
was data: reading_history existed but NOTHING wrote to it — the whole
stats feature was sitting on an empty table.
- CAPTURE (no client changes, no repo-boundary crossings): every
successful progress write through SaveProgress (KOReader + app + web
+ Kobo + offline queue — the one chokepoint) now records one
reading_history session row: span = the interval since the PREVIOUS
write instant, pages = positive page delta when both sides know the
page index, marker rows for first contact. Spans over 2h are kept as
zero-second markers (overnight suspend→resume, offline flushes must
not inflate totals) — the honest 'sync cadence, not a stopwatch'
caveat stays. device_id only when a REAL device row matches (the
web/app progress path historically stuffs the USER id into DeviceID
— probe instead of trusting it; FK violation would silently drop
every capture). Source preserved in device_metadata.
- API: /api/stats/summary?range=7|30|90|year (totals, zero-filled
daily arrays, streaks, most-active day/weekday, top books, library
breakdown — one call), /api/stats/this-week (widget: rolling 7d vs
previous + streak), /api/stats/books/:id (lifetime + recent
sessions). All bucketed in the user's timezone; finished ≡ any push
≥0.995; visible-library lens enforced at source. Legacy
/api/analytics/* kept untouched. Range windows anchor end-of-day to
the START of today (blasting the clock time into tomorrow resolved
'today' a calendar day late — found by the streak test).
- /stats page (sidebar Stats between Progress and Devices, previously
/analytics was an orphan): range selector, five stat cards, two
Chart.js v4 charts (daily minutes bar, pages/day line — chunked via
dynamic import so main.js stays untouched), per-library breakdown,
most-read list, empty-state explainer. /analytics now 301s to /stats.
The orphaned analytics.templ + analytics.ts removed (their containers
and the page JSON never matched; the page was astub).
- Dashboard 'This Week' widget (server-rendered, suppressed on service
failure) and the book detail 'Your Reading Stats' card (lifetime
totals of the viewer's own sessions) complete the tiers: per-user
default, per-library breakdown inline, per-book on the page,
per-system/admin aggregate parked as before.
Tests: statistics_test.go (capture semantics at the chokepoint,
tz/day buckets, zero-fill, streak math, finished detection, range
validation, auth gate, widget, per-book lifetime). Full suite PASS.
From the smart-collections build: templ emits quoted attribute values
literally (whole-attribute expressions required — the rules page
shipped '/api/collections/{ collection.ID }' until proven via request
logs); SettingsRegistry only reloads through the settings API (direct
SQL writes look ignored); and user collections must never route through
the dashboard-section path (query_type='filter' is a schema DEFAULT,
not a query type — the 7ce7321 static-detail regression).
Type is DERIVED, not stored: a saved ruleset makes a collection SMART,
no rules = STATIC (stored collection_items links, unchanged behavior).
No schema change — collections.auto_assign_rules JSONB already carried
the config; user's decision 2026-10-14 (handoff: live-at-render, not
refresh-on-scan; create-time Static/Smart toggle; v1 NO hybrid).
- Rule gains an enabled flag (absent = enabled, legacy rules unaffected).
- Rule CRUD: POST/PUT(DELETE) /api/collections/:id/rules[/:ruleId] — the
survivor dead effort of the rules-builder UI finally has real
endpoints (rules were never editable after creation; nothing ever
re-evaluated them). First rule converts a user collection to smart;
removing the last reverts it to static. UpdateCollectionRulesJSON
edits ONLY the rules doc so rule CRUD cannot clobber metadata.
- Live evaluation: candidates from GetVisibleLiveMediaForRules —
visible-library set (family lens enforced at source) + archived IS
NULL AND missing_scan_count = 0 — any enabled rule matches, title
order. Zero refresh hooks by design: always current by construction.
- Membership mutations (add/remove/bulk-add) are 400 on smart — v1 no
hybrid; collection_items.excluded stays the reserved future hook.
- Rules builder page (/collections/:id/rules) first time ever routable;
templ whole-attribute interpolation ("{ expr }" inside a quoted attr
value is literal text, as theme-{ user.Theme } silently proved).
- UI: create modal Static/⚡Smart cards (smart create redirects into the
rules builder), ⚡ badge + real per-card counts on the grid (smart =
live matches, static = stored-link count), smart detail renders
matched books with no hand-pick actions + an Edit Rules button.
- API: GET /:id gains is_smart + live books; GET /: keeps the Android
app's default exclusion, include_auto=true surfaces smart rows with
live counts; TestRules previews against the SAME candidates the live
render uses (visible + live, was raw first-1000).
- FIXED pre-existing regression from 7ce7321: static collection detail
with NO library_id returned zero books — QueryType != '' caught user
rows ('filter' default) and routed them to dashboard sections that
never contain them; now only real system query types go there
(handlers + SSR). App-visible symptom: /api/collections/:id without
library_id came back books:[] for every static collection.
Tests: collections_smart_test.go (conversion, disabled rules, mutation
guards, revert-to-static, validation, system-collection prohibition,
include_auto semantics). Full offline integration suite PASS.
GET /api/auth/password-rules (public, pre-auth) exposes the ACTIVE
complexity config from the settings registry; password_validation.ts
now fetches it and generates the checklist rows per active rule —
inactive requirements no longer linger as fake tasks, and the min
length mirrors the admin's setting instead of a hardcoded 8.
Surfaces: register, setup, admin create-user (Users page), profile
password change (self + admin variants, now gated on checklist pass;
misleading minlength=6 dropped — the server was always the enforcer).
Verified live: flip min_length=12 / number+special off via the
settings API → endpoint + register/profile/admin checklists re-render
row-for-row; enforcement follows (short rejected, rule-valid accepted
without digits/specials); restore defaults → checklists return.
The sed that removed the verify-guidelines lines also deleted the
.PHONY line (it contained those words), so 'make release' saw the
./release wrapper script as an up-to-date target and silently did
nothing. Restored with the verifier targets omitted.
Three changes from the network-drive audit:
1. Delete-ordering fix. Deleting a media item cascades its annotations,
so the delete must never proceed while the item's library storage is
unreachable — the DB row died, the file survived as an orphan, and
the next scan re-imported it bare (annotations gone forever).
resolveMediaStorage stats the library's folders: reachable → delete
proceeds and the file is removed by ABSOLUTE path (the old code
passed a library-relative path to os.Remove, which never resolved —
file deletion has been a silent no-op); every folder unreachable →
409 with 'storage unreachable', row and annotations intact.
2. Latency: resolveLibrary stats every library folder on every SSR
render; folder reachability is now cached with a 30s TTL.
3. README: Network Storage section documenting the outage hysteresis
(mark → archive → auto-restore), the 90-day retention purge
(ARCHIVE_RETENTION_DAYS, 0 = never), refused deletes while storage
is down, and watch mode not firing on network mounts.
Integration tests: storage-down bulk delete refuses with the row
intact; real-file delete removes the file from disk.
PROJECT_GUIDELINES.md had drifted out of reality (podman-only rule
while the stack runs docker; verifier scripts wired to Makefile targets
nobody runs). Trashed it, its two verifier scripts, and the Makefile
targets; the rules that still matched reality (docker runtime, schema
in place until release, server binary only via Docker, services layer,
integration-test pool rules) are absorbed into the new AGENTS.md.
The library-visibility family-lens rule now has a real home:
- AGENTS.md: the standing rule agents auto-discover (new media
endpoints MUST thread the visible-library set; 403, never silently
pass; history of the 2026-10-13 leak).
- docs/developer/library-visibility.md: the full explanation (helpers,
SQL pattern, per-item content gates, series per-library), linked
from the developer docs index.
Stale doc links updated (docs index, contributing, development).
Field-by-field audit of the profile surface (set → save → reopen →
read back, driven through the emulator browser via CDP):
- getTemplateUserWithTheme (every SSR page + the /profile self-edit
form) never mapped FirstName/LastName/CreatedAt — stored names
rendered as empty inputs. Now mapped (timezone was already mapped).
- Admin profile-modal Timezone mapping fixed earlier this round held:
all seven fields verified set → 200 + HX-Refresh → reopen → correct
values, DB cross-checked.
- Role promote/demote via the card select verified against the DB (the
PUT answers HX-Refresh, so the page reload shows the new role).
- Admin password reset verified end-to-end: new password logs in, old
one 401s, original restored.
- admin_library folder-remove hx-vals was string-concatenated JSON —
a folder path containing a quote or backslash made it invalid and
the remove button silently died under htmx 2 (the same class as the
profile-save bug). Now built with json.Marshal behind the js:
prefix. Verified the generated attribute on the live page.
- Other hx-vals/hx-headers sites audited: constants and uuids only
(safe charset), left alone.
The admin profile-modal route hand-rolled its templates.User without
the Timezone field, so every selected?={ user.Timezone == … } in the
timezone select was false and the first option (UTC) rendered selected
for every user on every open. Saves worked fine — the modal just never
displayed what was stored, which read as 'not saving'. The route's
GetUser query already returns the column; it just wasn't mapped.
Verified end-to-end in the emulator browser: modal opens showing the
stored zone, save → HX-Refresh → reopen shows the new zone.
The single-line card squeezed identity, created date, role select, and
two buttons into one row — on a phone the username truncated to t…,
the email to te…, and the created date wrapped vertically one fragment
per line (10- / 13- / 2026 / …). Restructured into stacked full-width
rows: identity (avatar + username + email), created on its own line
(a full date never wraps), a labeled Role row under a divider, and
Edit/Delete as side-by-side flex-1 buttons (thumb-sized on mobile,
right-aligned sm+). Last-admin protections render identically in both
controls. Verified on the emulator (Pixel 9, Brave): full names/emails
visible, no wrapping, modal + role controls functional.
Reproduced on the emulator via CDP: clicking Save Changes fired the
submit event, something preventDefault-ed it, and htmx aborted while
parsing hx-headers — 'Bearer ' + localStorage.getItem('token') is a JS
expression, and htmx 2 requires valid JSON (or a js: prefix) there.
The PUT never left the browser: zero htmx lifecycle events, no console
error, no toast. My earlier HX-Refresh fix was correct but unreachable
— the request died before it could ever be answered.
The header was htmx-1.x legacy and redundant: the browser already sends
the token cookie, which the auth middleware accepts. Removed hx-headers
from the profile form and both password-reset buttons.
Verified end-to-end in the emulator browser via CDP: Edit → change
timezone → Save → PUT 200, HX-Refresh reload, DB updated. The tab
reload mid-probe is the fix working.
The admin/user profile modal PUT succeeded but the response was a bare
JSON body swapped into a small div at the bottom of the still-open
modal — no close, no toast, nothing moved. It read as 'the button did
nothing' even though the save landed. HX-Request calls now answer with
HX-Refresh (re-render shows the saved values everywhere and closes the
modal) plus a visible confirmation. API callers keep the JSON body.
Silent-failure display already exists site-wide (toast.ts listens for
htmx:responseError), so failures were never the silent case.
The middleware's per-request user lookup (ghost-session guard) fetched
the full row and discarded it, then built the context stub without a
timezone — so every server-side conversion via the context user fell
back to UTC no matter what the handlers did. The stub now carries
Timezone from that same lookup (fetchLiveUser).
Live-verified: progress page renders 02:03 PM for a 14:03 New York
instant; an analytics session at 02:00 UTC buckets to 10-12 (10pm NY
the previous day) instead of 10-13.
The audit showed only 5 of ~11 time-display surfaces converted to the
user's timezone; everything else rendered pgx-decoded UTC. The server
owns conversion — clients (web/app/KOReader) receive display-ready
strings or machine-readable instants and never convert.
- Progress page: LastUpdated formatted in the requesting user's tz
(was raw UTC).
- Analytics: date-range defaults, day buckets, most-active-day, device
last-sync, and popular-book last-read all computed in the user's tz —
evening sessions no longer land on the next UTC day. Also fixed the
end-date boundary: the query bounds with <= and the end bound was
midnight, so the end date's own sessions (including 'today') never
appeared.
- Queue page: new created_at_display field (admin tz) rendered by the
template; created_at stays RFC3339 for API consumers.
- Admin hash-conflicts SSR bridge: detected date rendered in the
admin's tz.
- OPDS: .UTC() before the Z-suffixed feed timestamps (honesty fix —
correct today only because the session zone happens to be UTC).
- KOReader annotation datetimes deliberately untouched (device owns its
display).
default_timezone existed as a system setting (admin-editable, exposed
via SettingsRegistry.DefaultTimezone) but nothing read it at user
creation — the CreateUser query didn't even carry the column, so every
new user got the DB default UTC regardless. Both creation paths
(public register + admin create) now seed the user's timezone from the
setting, validated with time.LoadLocation and falling back to UTC when
unwired/empty/invalid.
Test-infra fixes surfaced by the run: TestUpdateProfileSelfEdit was
self-editing the SHARED admin (renaming it past the harness cleanup's
preserve-list, leaking a second admin row that broke the last-admin
tests) — it now edits a dedicated user with a unique name; the
last-admin tests get an ensureSingleAdmin precondition helper; the
websocket collection name is run-unique (non-test names persist in the
shared DB and UNIQUE(user_id,name) 409s on rerun).
The /register checklist was dead markup: initPasswordValidation() was
only ever called on the setup page, so the requirement rows never
updated and the Register button stayed disabled forever — web
self-registration was effectively broken. The register Alpine component
now initializes it.
The validator is parameterized (field/submit ids, confirm + username/
email checks optional) so the admin Create User form reuses it: confirm
field, live requirement rows, and the Create button gated on all checks
passing. Requirements mirror the server's passwordcomplex rules
(8+ chars, upper/lower/number/special — the hardcoded defaults; server-
tuned rule values are a future nicety, the server remains the enforcer
either way).
The md:-flipped table/card split still broke at tablet widths and with
the nav sidebar open — a viewport breakpoint cannot see the sidebar
eating content width. Replaced the table entirely with the same fluid
card list admin_library uses (flex + min-w-0/truncate identity block,
fixed action cluster, flex-wrap): it reflows continuously at every
width instead of stepping between two layouts. The duplicate-DOM-id
suffixing from the previous two-rendering approach is gone with it.
- The desktop-on-Android bug: 8 page templates (admin, admin_settings,
admin_users, admin_library, admin_archived, admin_hash_conflicts,
admin_processing_issues, profile) lacked the viewport meta, so mobile
browsers fell back to a ~980px layout viewport and rendered the
desktop layout. Added the same meta the other 21 pages already had —
partials without a head were skipped deliberately.
- Admin Users page: the create-user form no longer dominates the page —
it is hidden behind an explicit '+ Create User' button (Alpine
x-show/x-cloak, the site-wide convention) with a Close control.
- The user list is responsive: table on md+, stacked per-user cards
below. Role/delete controls are shared partials with DOM-id suffixes
so both renderings keep unique swap targets (last-admin protections
preserved in both).
The family-lens hardening: library_visibility was only enforced where
file bytes flowed (download/reader/OPDS) — metadata surfaces leaked
hidden libraries to any authenticated account, and registration was
open by default with no admin-side user creation.
- Series are grouped PER LIBRARY now (same name in two libraries =
two entries: manga adaptations / LN splits no longer merge);
/api/series aggregates only the requester's VISIBLE libraries and
/api/series/books REQUIRES a visible library_id (breaking contract;
web + app clients updated in the same change).
- /api/media-items, item detail, and the reader content endpoints
(chapters/panels/outline/THUMBNAILS — page images leaked unguarded
next to a gated GetPage) all enforce the visible set; resolveLibrary
no longer honors a hand-typed library_id outside the visible set.
- Annotation/note/bookmark mutations (update/delete/tombstone, sync
LWW paths incl. conflict resolution) carry ownership predicates:
another account's bare row id is a 404, not a silent delete.
- Admin user creation: POST /api/auth/users (AdminMiddleware) + a
Create User card in the admin Users UI — accounts stay creatable
with self-service registration OFF, which is now the schema default
(fresh installs; the first-admin bootstrap exception is unchanged).
- Tests: series contract re-pinned (per-library, visibility 403s),
new ownership + admin-create integration suites, OPDS all=1 test
now passes an explicit page size (shared-DB growth made the fresh
books fall off page 1), device-cap helper switched to the admin
endpoint.
The export/ directory holds database extracts (annotations, reading
positions, book metadata) pulled from production for debugging. The
contents are personal data and must never be committed.
scripts/build_torture_epub.py deterministically builds a small EPUB
exercising every CRE-xpointer to CFI conversion hazard observed in the
wild, one chapter per hazard class: multi-text-node paragraphs (verse
spans, inline markup), whitespace-only inline spans and comments,
drop caps and footnote anchors, br-separated poetry with speaker and
line-number spans, nested blockquotes/lists/tables/definition lists,
surrogate pairs and ZWJ emoji, Hebrew niqqud, Arabic RTL, combining
diacritics, invisible characters, HTML entities, CJK ruby, a 400-
paragraph chapter with heavy phrase repetition for search
disambiguation, and edge constructs (span-first paragraphs, empty
spans, adjacent same-tag spans, deep same-tag nesting).
Default output lands in the Ebooks library for real-device testing
via OPDS; the script takes an output path argument for use as a test
fixture.
TestManualCleanRepro replays a real device highlight export through the
same conversion path the sync handler uses, diffs the output against
the CFIs stored at ingest time, and computes a ground-truth oracle by
locating each selection in the flattened document itself. Classifies
every row as healthy / diff-base / backwards for both the converter
and the stored data. Skips automatically when fixture files are not
present on the machine (manual test, like cfi_verify_manual_test.go).
REPRO_VERBOSE=1 prints per-row detail; REPRO_EMIT_SQL=<path> emits
idempotent UPDATE statements that heal database rows written by the
old converter from the preserved raw device positions - used to
backfill dev and validated against production data.
toRangeCfi required stored start and end CFIs to share a base path and
otherwise fell back to a bare point CFI, which resolves to a collapsed
range and paints nothing - or painted from the wrong anchor after end
repair. A legitimate selection crossing inline elements anchors at two
different text nodes by definition, so requiring a shared base path
discarded correct endpoints.
Build the range parent from the endpoints' longest shared step prefix
(down to the bare spine step when needed) and emit each divergent
remainder per the EPUB CFI range grammar, epubcfi(parent,start,end).
Degenerate-end repair (missing end, or end before start) now only
applies when both sides terminate on the SAME node - equal remaining
step count with different step values is a valid two-node range, not a
degenerate pair; document order is verified for those instead.
Verified with foliate's own parser against the real chapter XHTML:
ranges resolve to exactly the selected text for same-node, two-node,
and repaired-degenerate cases.
KOReader highlights converted to wrong or unresolvable web CFIs
whenever a selection crossed text-node boundaries. Three compounding
defects: ParseCREXPointer discarded the text()[N] node index and
applied .offset as a global block offset; the end anchor converted
with an empty context (blind structural landing, zero verification);
and the block-containment check accepted wrong-node landings because
a selection's text always occurs somewhere in its own block. pos0 and
pos1 were converted independently, yielding mismatched start/end base
paths and backwards ranges - highlights stored but never painted, or
painted at the wrong location, in the web reader.
Add ConvertHighlightRange: both endpoints are anchored by locating the
selection text in the flattened document text (whitespace-normalized,
crossing block boundaries), with the device locators demoted to
disambiguation hints. A single document match yields a same-spine,
ordered start/end pair by construction. Falls back to the legacy
per-endpoint path when the text cannot be located, so behavior is
never worse than before.
Also fix the shared flatten stream: zero-width format characters
(ZWSP/ZWNJ/BOM) and whitespace-only inline spans (<span> </span>)
now act as word separators instead of fusing adjacent words, which
prevented text matching in real books.
Verified against 28 real device highlights from production: converter
output now matches the text-search oracle on all of them (previously
8/28 correct, 18 with mismatched base paths, 2 backwards).
The "full 2D" commit (5a0d9e0) added the tip aim but never removed the
y-band candidate filter from pdfAdjustHandleMove — vertical handle
drags still snapped to the boundary rect's own line, so up/down
movement was impossible (the user's original report). With the filter
gone: the end handle drags down a line and the extent extends with
clean text — verified with real touch on the emulator (down one line,
release, PUT applied: 6 per-line rects).
User report after the lifted-handle change: "I cannot go up or down,
just side to side" — a consequence of the line-band pin added with the
lift (the floating finger sits above the text; a band-constrained scan
ignored vertical movement entirely).
The endpoint drag is now full 2D: the finger position (plus the
~15px body-center-to-tip offset) maps to page fractions and the
nearest glyph re-anchors the grabbed end — up/down crosses lines,
side-to-side moves along them, clamped so the extent never inverts.
The boundary rect's own band constraint is gone; sliver rects remain
structurally impossible (rects rebuild from the char-table span as
per-line unions).
Direct-fraction probes verified: (0.4, 0.098) → mid line 1; (0.2,
0.117) → line 2 head; (0.72, 0.117) → line 2 mid — spans cluster into
the right per-line boxes. The dot-listener plumbing (pixels → map →
fractions → scan) verified by instrumented probe on the same load.
Final desktop finger pass pending (emulator input pipeline wedged).
User report: the teardrop tip touched the letters. Raised the handle
boxes 8px (tip now rests at the line-box top, ~4px above the glyph
tops) — matches the Android handle spacing the app leg shipped.
User report: the handles covered the highlighted text. The dots now
hang ABOVE the extent's boundary corners (teardrop tip down at the
first rect's top-left / last rect's top-right), so the highlighted
line stays readable.
Same treatment as the app leg: the nearest-char scan in the endpoint
drag is band-constrained to the boundary rect's own line (the lifted
finger floats ABOVE the text — a whole-table scan re-anchored onto the
line above). Cross-line changes go through re-selection as before.
User report (desktop Brave, fresh private window): the endpoint handle
"jumped several lines down and stuck". The instrumented drag exposed
the chain:
1. THE bug: the endpoint handleMove API takes PAGE FRACTIONS, but the
dot's pointermove listener passed raw e.clientX/clientY PIXELS.
Scanning for the nearest glyph to (296, 170) in a 0–1 fraction space
picks the bottom-right-most glyph — the extent leaped to the page's
end and never moved again (stuck). Engine-independent: failed
identically on Chrome/Brave/Firefox desktops; my synthetic tests
masked it by passing pre-converted fractions. handleMove now maps
host px → fractions internally (one contract).
2. pdfPageMapping mixed coordinate spaces: the iframe-internal canvas
rect with the host iframe rect. Whether that combination is valid
depends on the engine's cross-document getBoundingClientRect
semantics. Replaced with the page WRAPPER element (the iframe's
parent — the same box the fork's rect-annotation overlay fills),
measured in a single host context. Handles now align with the
painted highlights by construction.
3. Char-table hygiene: per-char boxes are LINE-HEIGHT boxes (~one line
gap tall). Dedupe at a full box height ate the line-below char in
the same x-column (silent character loss); clustering at full box
height never split lines (merged 0.45-tall "line" boxes). Dedupe
now tolerates 0.75 of the SHORTER box; clustering breaks at 0.6 of
the char's own height (inter-line gap ≈ 1.0× breaks, baseline
variance ≈ 0.1× stays).
Verified end-to-end (CDP synthetic pointer path): drag to (0.72, 0.117)
→ span extends line 1 → line 2, 2 per-line rects saved, clean
multi-line text. Direct-fraction probes and the pointer path now agree
on the same load.
User report (desktop): the endpoint handles wouldn't move.
renderPdfAdjustHandles rebuilt the handle layer via innerHTML="" on
every preview frame — the first drag move triggered a repaint that
DESTROYED the dot being dragged. A removed element loses pointer
capture, so real mouse/touch drags died silently after one pixel; the
pointerup never reached the detached dot either, so nothing saved and
the session preview reverted — the same mechanism behind the earlier
"resets to the original selection" report. CDP tests masked it because
synthetic events dispatched directly at the detached node still ran its
listeners.
The two dots are now created once per session and every later call only
repositions them (style updates move nothing the listeners live on).
Grip-era code had this reuse pattern; the endpoint rewrite regressed it.
Real-touch verified on the emulator (adb-injected drag): the handle
tracks, the release saves a multi-line extent with clean recomputed
text (3 rects, PUT applied).
The static handler sent no Cache-Control, so browsers fell back to
heuristic caching (10% of age since Last-Modified) and kept serving a
stale reader.js for hours after a deploy — fixed web bugs reported as
"not fixed". no-cache forces revalidation every load; the filesystem
Last-Modified/If-Modified-Since pair keeps unchanged loads cheap.
Found while closing out the PDF adjust v2 session: the user's Chrome
was running a pre-endpoint bundle, which read as "the web has not
been fixed".
Multi-line partial extents ("line 1's tail + the first words of line
2") were inexpressible with one-rect-per-drag grips, and the vertical
drags produced sliver rects whose text recompute swept fragments. The
web's re-selection uses what creation already uses: a selection made
during a rect-edge session is intercepted in checkPDFSelectionFor —
rects rebuild from the selection's per-line client rects, text re-reads
via the deduped glyph boxes, saved immediately (grip release-to-save
parity), and the create popover is suppressed for that gesture.
Vertical grip drags clamp to the rect's OWN line box (band ±quarter
height) — multi-line changes go through re-selection, slivers are gone.
Also verifies clean under the duplicated text-layer span sets (the
fork's pdf.js render never clears .textLayer — open bug): the text
recompute dedupes by glyph overlap.
CDP-verified end-to-end: session → multi-line selection → 6-rect
extent + clean text persisted (PUT 200); create popover correctly
suppressed during the session.
User report: extending a highlight doubled letters into the recomputed
selection text ("aauuthenticcity off ... ruunss iinnto"). The exact-
position dedupe key (toFixed(3)) only collapsed duplicate glyphs whose
re-render copies landed inside the same rounding bucket — pdf.js
re-render copies can sit SUB-PIXEL SHIFTED, so per character the jitter
sometimes straddled a boundary and both copies survived, partially.
Dedupe by overlap instead: a glyph is a duplicate when a kept glyph on
the same line (y0 within half a glyph height) overlaps more than half
the narrower glyph's width. Distinct glyphs — including tight pairs —
never reach that; shifted duplicate copies always do. The snap tables
share the source, so their boundary lists are clean too.
The root cause remains the fork's pdf.js render() appending a fresh
span set without clearing .textLayer (open bug, fork-cycle fix); this
is the defensive layer that makes the adjust feature correct regardless.
Repaired the affected highlight's stored text with the fixed recompute
(verified: the same live duplicated DOM now yields the exact line text
with zero doubling).
fetchBannerAnnotations published the rebuilt rows unconditionally, with
a failed endpoint's list defaulted to empty — so one failing request
(a single handler 500, an auth blip on one call) silently wiped that
annotation kind from the banner while the other kind stayed, and the
empty state could claim "No notes or highlights on this book yet" over
real data. A full outage was already safe (the thrown fetch takes the
catch path, which keeps previous rows and toasts) — the partial case
was the hole, spotted while mirroring the app's fetch-wipe fix
(bookhoard-app 07bbb4c) onto the web.
Both responses must now succeed before the rows are published;
otherwise the previous rows stay and the next banner open retries
(bannerLoaded only sets on success). Verified on the emulator: banner
renders 3 rows, a full outage keeps them with bannerError recorded,
and a post-recovery refetch re-renders from live data.
The left-edge drag kept the rect width instead of anchoring the opposite
edge, so shrinking from the left silently dragged the right edge along
and the recomputed text grew words at the far end (found by the Android
device test of the app leg). Same fix for the top edge / bottom pin.
The end-edge branches were already correct.
The reflowable adjust (objective 9) had no fixed-layout counterpart: the
popover's Adjust button relied on adjustContext, which is only stashed by
the reflowable show-annotation path, so PDF rows showed a button that
silently did nothing. PDF highlights persist as page-fraction rect
payloads ({v:1,page,rects}), so their adjustment is geometry, not range
re-anchoring.
- startAnyAdjust routes the popover button by format; startPdfAdjust
runs a rect-edge session: 4 grips per rect (all edges, one rect per
drag), live preview through the renderer's own addRectAnnotation
(replace-by-key — no fork change), commit via saveHighlightChanges.
- Snap vocabulary mirrors snapBoundary: maximal letter runs are atomic
tokens (start edge takes the word start, end edge the word end), each
punctuation glyph is its own token with both edges grabbable (a period
or line-break hyphen can be taken or shed precisely). Horizontal edges
snap to line top/bottom bands (own line + neighbours).
- selection_text is recomputed from the glyph boxes inside the final
rects (reading order, lines joined with spaces) so the stored text
stays honest with the extent.
- Cancel paths mirror the reflowable session: page turn (renderer
relocate) discards and restores; a committed session skips the restore.
- getContents() from the fixed-layout renderer carries no index — page
iframe docs are mapped through a WeakMap filled by the view's load
events. Pointer mapping goes through the canvas's post-transform rect
(the dpr denominator lesson) via the fork's own convertPoint ratios.
- pdfCharBoxes dedupes by box key: pdf.js re-renders append a second
span set into the same .textLayer without clearing, and undeduped
glyphs doubled the recomputed text.
Device-verified (emulator Chrome + CDP): popover route, grip placement,
word-atomic shrink ("But this" -> "this"), extension to the exact
line-break hyphen boundary (0.8777), far-drag clamp at the last token,
page-turn cancel, and the reflowable adjust still routing to handles.
The guard returned c.JSON(401, ...), whose nil error told echo the
SuccessHandler succeeded — the pipeline then continued into the route
handler, which type-asserted the never-set "user" context value and
panic-recovered as a 500 appended after the 401 body. The status code
happened to survive (first write wins), which is why the pin test
passed, but every ghost request also executed the target handler with
no user — the exact unauthenticated-execution class the guard exists to
prevent.
Return echo.NewHTTPError(401, ...) instead: non-nil aborts the chain
and renders {"message":"user_not_found: ..."}; the pin test now asserts
that shape. Full suite: 219 groups, zero failures.
The JWT middleware validated signature + expiry and fabricated the
request user purely from token claims, so a signature-valid token for a
DELETED user kept full API access until token expiry — observed live on
hardware (a phone session survived the test suite's user-table
recreation and kept serving its true empty sets; open handoff bug 1).
An ensureUserExistsMiddleware already existed but was only wired on the
frontend SSR group (frontend.go) — every /api group used bare
createJWTMiddleware. The existence check now lives INSIDE
createJWTMiddleware's SuccessHandler so every protected group (SSR and
API alike) gets it:
- After claim validation, GetUser by the token's user_id; a missing row
clears the token cookie and answers 401 user_not_found for API calls
(the app's TokenAuthenticator then re-logins with stored credentials)
or redirects to /login?session=invalid for page requests — same
shape-aware split the error handler already uses.
- Cost: one primary-key lookup per authenticated request (~0.2ms local);
instant revocation, no TTL semantics. The redundant SSR-only
middleware is deleted (its redirect behavior is preserved inside the
guard).
The guard exposed two processing-issues tests that were unknowingly
RELYING on ghost semantics: they nested a second setupTestServer whose
bootstrap deleted the first setup's regular user, then asserted 403
from the dead token. De-nested: a createLibraryDirect helper inserts
the library under the live setup's user, no second bootstrap.
New TestDeletedUserTokenRejected pins the contract: token works while
the user exists -> 401 user_not_found after deletion. Full suite: 219
groups, zero failures.
Handler.UpdateUniversalProgress (handlers/progress.go) had no route
registration anywhere (router/progress.go only wires the history route
to the scanner handler) and no callers — a legacy progress-write path
that bypassed the verify/heal pipeline entirely. Deleting it removes
the last bypass-shaped writer that could poison progress rows without
verification; conflict-resolution apply remains the only deliberate
direct write (user-approved winner rows).
Pure deletion: 115 lines, no route, no callers, no behavior change.
The verify/heal pipeline verified only INBOUND context-bearing writes;
stored-row coherence was never enforced. SaveProgress seeds missing
fields from the existing row, so every percentage-only write (KOReader
low-confidence conversion, KOReader no-pointer payloads, Kobo markup
ReadingSync / analytics / sync-from-server, locator-less queue entries)
silently re-paired its NEW percentage with the PREVIOUS writer's stale
CFI+context — the incoherent row shape that fed the client-side
far-anchor restore floor (and, via the read-side anchor handles, could
even hand a first-party client the OLD position's scroll address).
KOReader's converter already declared the right intent ("storing
percentage only — the row's percentage restores approximately"); the
seeding betrayed it. SaveProgressRequest gains ClearLocator, set by the
percentage-only call sites, and the seeding rule honors it: stale
epubcfi/context/character_offset clear (the upsert assigns columns
directly, so NULL sticks) unless the request carries its own. A
request-carried context survives so the verify/heal pass can still mint
a fresh anchor from it; the HTTP merge path (field-absent PUTs
preserve) and all anchor-bearing writes are untouched. The fixed-layout
branch already cleared locators unconditionally — unchanged.
Consumer audit: percentage — the only thing every sync round-trip
exchanges — is untouched on all paths; KOReader pulls on locator-less
rows already support the nil-CFI shape (builder guards
progress.Epubcfi.Valid); first-party restores go percentage-first
instead of floor-fighting a stale anchor.
Tests: TestProgressWeb_LocatorCoherence pins clear-on-declared,
request-carried-context survival, and merge-path preservation. Full
suite with the change: 218 groups, zero failures.
Closes the remaining integration-suite failures (handoff open bug 3),
line by line:
- Fixture family (the upsert equivalence): tags_filter_test
("Science Fiction" found 1 of 3; "Fantasy" 404'd on zero results),
media_item_isbn_test (normalization read back the first case's ISBN
everywhere; edge cases saw a nil ISBN; pagination collapsed 5
creates onto 1 row), search_unified_test (its 3 books were actually
1 row + two no-op creates) — all POSTed a constant file_path into
one library while CreateMediaItem upserts ON CONFLICT
(library_id, file_path). Every fixture now uses unique per-item
paths, the suite's universal convention elsewhere. TestTagsFilter
confirmed same-family: counts came right with no second bug.
- New TestMediaItemCreateDuplicatePath pins the honest-create
contract from da13898: 201 for the first create; 200 + duplicate
flag + the EXISTING row (never a 201 lie), second payload not
overwritten.
- library_browse_test: the admin subtest browsed the container's
/tmp and required it non-empty — env-coupled (empty /tmp → "was
[]"). Now browses a self-created t.TempDir with two subdirs and a
decoy file, asserting ONLY directories are listed.
- series_integration_test: three stale assertions re-anchored to the
shipped contracts, each with a ground-truth comment: /dashboard/
sections key identity on the collection NAME (BuildSections ID =
CollectionName; the web/app reorder + hidden prefs persist those
strings) — the kebab query_type is the storage key and is not
exposed; /api/series/books is name-ONLY (the app client and the
web SSR page both send name only; library_id is ignored, never
validated); /api/series is optional-scope BY DESIGN (GetDistinct-
Series: narg IS NULL OR library_id = narg aggregates across
libraries when omitted). The former 400-on-missing-library_id
expectations asserted validation that never shipped.
- gofmt rider on three untouched-in-this-arc test files
(dashboard_integration, search, test_helpers).
Full suite with -parallel=1: 217 groups pass, zero failures.
The create endpoint shared the scanner's upsert (ON CONFLICT
(library_id, file_path) DO UPDATE SET updated_at = NOW() RETURNING *) —
so a POST naming an already-registered file returned 201 CREATED with
the OLD row while silently discarding everything the caller submitted.
Nothing depended on that lie: the app only GETs/PUTs media items, the
web only searches/purges/deletes, KOReader never calls it — the
endpoint's create path had zero production callers.
Split the API path from the scanner's idempotent upsert, which is kept
verbatim in CreateMediaItem (re-scan dedup remains its safety net):
- CreateMediaItemIfNew: same insert with ON CONFLICT DO NOTHING — the
loser of a create race writes nothing and returns no row.
- GetMediaItemByLibraryAndPath: fetch the existing row for the
duplicate response.
- Handler branch: on pgx.ErrNoRows → 200 OK with
{"duplicate": true, "message": ..., "item": <existing row>} instead
of a 201 pretending it created something. Real creates still answer
201 with the item as before.
Bulk-import shape (the reason for 200+flag over 409): a re-run after a
mid-import failure is harmless and countable — each response tells the
caller exactly which of the two things happened.
Verified by the full integration suite: 217 groups pass, zero failures,
including a new pin of the duplicate contract (same file_path twice in
one library → 201 then 200/duplicate:true, second payload not
overwritten).
Born dead in e7ce0cb (2026-04-03, backend reader infrastructure):
the "Phase 2.5" plan assumed MOBI/AZW3/DOCX/RTF would need server-side
parsing, so ParseEbook was written as the endpoint that would do it —
but it only ever returned 501 "not yet implemented", and no route in
router/reader.go ever pointed at it. No caller in Go, templates, or
web JS/TS ever existed either.
The eventual architecture never needed it: the web reader hands the
raw file to foliate-js and parses client-side (MOBI renders via the
fork's mobi.js — verified live; DOCX via mammoth; RTF/PDB shipped).
By now it was also stale — it 501'd DOCX/RTF as "not yet implemented"
while both work, and offered an AZW3 path that was dropped entirely.
Pure deletion: no route registration, no references, no tests, no
behavior change. go build ./... clean.
TestProgressWeb_KoboIntegration 500s'd on every markup with a
last-read-place bookmark: the panic-recovered stack was a nil pointer
dereference in LibraryService.ResolveMediaPath (s.db deref), reached
from KoboHandler.convertKoboCFIToStandard during kepub CFI conversion.
Two-layer bug:
1. Test harness (root cause): the router.Config built in
test_helpers_test.go omitted LibraryService, so router/sync.go's
SetLibraryService(cfg.LibraryService) stored a nil
*services.LibraryService in the interface field. Production wiring
(main.go:246) was always correct.
2. Latent trap (hardening): `h.libraryService != nil` at the
convertKoboCFIToStandard call site can never catch a typed nil —
an interface holding a nil *LibraryService is non-nil. ResolveMediaPath
now degrades with a "library service not configured" error instead
of panicking on a nil receiver; callers already handle the error
by skipping CFI conversion.
Harness-only fix, zero production behavior change: the endpoint now
returns a clean 200 with the position saved wherever kepub conversion
is unavailable instead of a panic-recovered 500.
Verified: TestProgressWeb_KoboIntegration passes all three subtests
(401 unauthenticated, ReadingSync→last-read-place percentage
preservation, last-read-place without prior ReadingSync) and on
immediate rerun — previously it failed every run.
The whole-body "should not display Manga Type / Scan Info" assertions
could never pass: every book-detail page for an admin carries the
edit modal, whose form labels ("Manga Type", "Scan Information") and
option texts ("Yes (Right to Left)") share vocabulary with the
metadata display block. This was pre-existing rot — the test never
ran to completion before today because an earlier panic killed the
process first.
Replaced with value-level checks that only the display block could
produce ("yes and right to left", "B&W"): absent for plain ebooks and
unknown-manga-type items, matching the templ guards in
book_detail.templ. Emoji-icon asserts dropped from the RTL/Story-Arc
positive steps — the redesigned comic badge renders text chips
without icons.
Symptom: repeated full-suite runs poisoned the dev database with
leaked rows (duplicate Prefs Scope libraries, colliding device names,
saved-filter name conflicts on rerun) because tests created rows but
never deleted them, and stale-binary runs left rows the new cleanup
could not match.
- test_helpers: library creation now registers t.Cleanup DELETE (the
leak root cause — createTestLibraryWithFolder never cleaned up).
Suite-wide cleanup passes for libraries/collections/devices/
saved-filters run both at setup (pre-pass against legacy residue)
and in t.Cleanup (no per-run residue). Preserve the admin's real
libraries; delete only convention-named test rows.
- setup seeds canonical system collections every run keyed on
query_type (idempotent — heals renames/stale tombstones), moved out
of the preserved-admin branch so it runs for every suite start.
- dashboard_integration: scope-testing library renamed into the
test-row convention; ordering assertions on the restored
"Want to Read" collection preserved.
- filters_test: filter names moved into the convention (rerun-safe —
name-uniqueness 409 no longer trips second runs); five .(string)
type assertions panic-proofed with require.True guards (a panic here
used to kill the whole test process mid-run, which is why a stack of
"never ran" failures surfaced at once later).
- device_test: device names moved into the convention (ILIKE %test%
hit them; duplicate-device-name conflicts gone).
The old model hard-deleted system collections and re-created them at
the tail, resurrecting stale dedups and breaking ordering. Redesign
(user-directed B-final):
- schema.sql: collections.deleted_at TIMESTAMPTZ (idempotent ALTER —
the whole schema is re-applied on every boot via executeSchema).
- queries.sql: every collection list/read query filters
deleted_at IS NULL; new GetSystemCollectionByQueryType :one
(unfiltered, sees tombstones), TombstoneSystemCollection :exec,
ClearCollectionItems :exec, RestoreSystemCollectionRow :one
(in-place metadata reset + un-tombstone; collections have no
updated_at column — ResetUpdatedAt was a 500 caught by tests).
sqlc regenerated.
- collection_service.DeleteCollection: system collections are
tombstoned (preserving position/ownership); user collections keep
the hard delete. No ErrNoRows to the client (delete is idempotent).
- dashboard_service.RestoreSystemCollection: resolve by query_type;
found → in-place restore (ClearCollectionItems + metadata reset,
un-tombstoned); missing → provision fresh. "Want to Read"/etc.
stay at their home positions forever.
- handlers/collections.CreateCollection: name-taken is 409, not 500.
- handlers/auth.CreateDefaultCollectionsForUser: idempotent by
(user_id, query_type) — update-in-place when a row for that query
type exists (heals renames/tombstones), insert when missing; this
also fixes the duplicate-system-collections regression at source.
The drawer was the only top-bar tool without a keyboard shortcut, so the
shortcuts & gestures help modal had to describe it as '📝 button'. Bind
plain 'a' to toggleBookmarks(), following the same pattern as the
existing t/s/b letter shortcuts.
- reader.ts: new 'a' branch in handleKeydown, guarded against
Ctrl/Cmd/Alt so Select All (a common reader action when selecting
text to highlight/copy) is never intercepted; typing guard already
protects note textareas, search box, and selects. Works in both the
host page and book-content iframes since both keydown listeners route
through handleKeydown.
- reader.templ: help modal row now shows <kbd>a</kbd> like the other
letter shortcuts; 📝 button tooltip gains '(a)' to match its
siblings (b), (/), (s), (?).
- reader_templ.go: regenerated.
- BUG: startBannerEdit extracted the note by splitting row.rich — the
RENDERED HTML — on \n\n, which the concatenated markup never
contains, so highlight-note dialogs always opened empty. Both row
kinds carry the raw note on bannerSources; use that lookup.
- Expanded body now splits at the highlight→note boundary: the
selection paragraph (left border in the row's highlight color), a
divider + pencil NOTE label, then the note markdown — instead of the
two texts flowing together.
- The note-indicator pencil (corner badge) now shows on ALL note rows,
not just highlights with attached notes.
Verified in headless Chromium: badge on note and highlight rows;
expanded highlight shows the bordered selection, NOTE label, and note
markdown; the highlight edit dialog prefills with the note text.
The test helper createDefaultCollectionsForUser carried its own
kebab-case copy of the default system collections — drifted from the
canonical Title-case list the registration seed and
RestoreSystemCollection use. Every suite run against the shared dev DB
re-created a kebab set beside the canonical one (the duplicate sections
on the customize screens); the dev DB is cleaned and the helper now
uses the canonical list.
Two stale expectations corrected while re-running the suite:
TestGetSections_EndToEndFlow asserted the kebab ids (now asserts the
Title-case names — section ids ARE collection names), and
TestGetSections_MissingLibraryID expected a 400 the handler never
implemented — absent library_id is the All Libraries view by design
(now asserts 200 + aggregated sections).
The All Libraries entry in the library switcher could render the
cross-library dashboard but rejected its own settings: the modal
refused to open without a library ('Select a specific library...') and
a save sent library_id: '' (server 400). The server now stores a
global scope (separate table; null/absent library_id, empty string
still 400), so:
- openDashboardSettings allows the All view and fetches prefs without
the library param (reads the global row; previously a loose NULL
OR-clause matched an arbitrary library's row).
- saveDashboardSettings sends library_id: null from All Libraries.
- The settings modal gains a scope label (#dashboard-scope-label)
showing which library — or All Libraries — a save will customize
(user requirement: always know the scope).
Verified end-to-end in headless Chromium: reorder on All Libraries →
save (null) → reload → modal re-opens with the persisted order and the
All Libraries scope label; per-library save lands in its own row
without touching global (SQL-checked).
The dashboard's All Libraries view could render sections across every
library but could not persist its own preferences: the PUT required a
UUID library_id, and the prefs GET-without-library_id matched an
arbitrary library's row through a loose NULL OR-clause.
Storage: a dedicated user_dashboard_preferences_global table (one row
per user, user_id PK → users CASCADE) instead of a NULL/sentinel
library_id — chosen over both because (a) the column's FK to libraries
rules out a sentinel UUID without dropping the constraint or minting a
fake library row, and (b) intent becomes structural: only the explicit
global path writes that table, so a buggy or outside-the-API write can
never masquerade as global. System and user collections are
user-scoped, so the order/hidden arrays are library-independent and
valid in this scope.
API: PUT /api/dashboard/preferences binds library_id as *string —
JSON null/absent = global, empty string = 400 (a lost value must stay
loud, not silently clobber the global scope), valid UUID = per-library
(unchanged). GET without library_id now returns the global row instead
of an arbitrary one; the per-library query's dead NULL OR-arm is
removed. GET /sections keeps requiring library_id for per-library
views and aggregates across libraries when absent (unchanged).
Tests: global save/update-dedupe/read round-trip, empty-string 400,
global-vs-per-library independence. Two suite failures are the
handoff's documented pre-existing rot (duplicate system collections in
the dev DB; a sections test expecting a 400 the handler never
implemented) — reproduced and confirmed unrelated to this change.
Live-verified on the dev stack: null save 200 x2 with a single row
updated, empty string 400, global read carries the second save.
Highlights with an attached note get a small edit-note stroke SVG
(same note-pencil path as the selection popover's Note affordance) in
the card's upper-right corner — themed via var(--text-secondary),
pointer-events: none so taps pass through, header gains pr-8 on those
rows so wrapped text never runs under it. Placement matches the
Android banner's corner badge (user preference: never left of text).
Regenerated annotations_banner_templ.go from the repo root.
Tailwind preflight zeroes p/blockquote/list margins globally, so notes
rendered through renderNoteHtml collapsed flat: blank lines between
paragraphs vanished, blockquotes were indistinguishable from body text,
lists had no bullets/indent, code blocks had no padding. The parser was
always correct — the loss was CSS at the x-html display sites.
One shared scoped block (.note-rich, in @layer components so it
outranks base preflight): p+p margin-top, blockquote bar + indent +
secondary color, list markers + padding, pre/code padding + themed bg,
modest heading sizes, underlined links, table borders, em-based so it
scales with each site's text-xs/text-sm context. Applied at all three
display sites: reader notes-drawer rows (replaces the unused
reader-note-rich hook), the 📝 highlight note line, and the banner's
expanded body.
The regenerated _templ.go files also catch up two stale .templ changes
that had shipped without regeneration (the selection popover's Adjust
button, the banner delete ternary) — the Docker build regenerates at
image-build, so the served site always matched the sources.
Verified live (headless Chromium against the dev stack): .note-rich in
the served stylesheet; p+p computed marginTop 9.6px, blockquote 3px
solid border-left, ul disc + 22.4px indent, pre padded + themed bg.
Objective 9 (web half). The selection popover's edit mode gains an
Adjust button: the overlay's show-annotation event stashes the live DOM
range, start/end teardrop handles render over it (host layer above the
iframe), and pointer drags map into the iframe's caretRangeFromPoint.
Snapping: land next to punctuation → the boundary sits exactly there;
land inside a word → the word's edge in the drag direction; whitespace
trims toward the adjacent word. A live preview repaints through the
overlayer under a scratch key; relocation (page turn) cancels safely.
Release recomputes the range CFI + collapsed-end CFI (view.getCFI) and
saves through the popover's own saveHighlightChanges path (PUT +
remove-old/add-new overlay dance), keeping selection_text and the
anchors in lockstep.
Verified live E2E in Brave: extent extended by one word with the
comma/space snap behavior, saved to the server row, handles cleaned;
all snap paths unit-exercised in-page (word end/start, punctuation
exact on both sides, whitespace trim); the test highlight was restored
to its original extent through the same adjust flow (punctuation-precision
drag back to the comma).
The book-detail banner's highlight rows gain Delete (tombstoned
server-side, restorable from the Deleted modal — same as the reader
drawers already offered). The deleted-annotations modal now says the
retention in plain language — 'kept for 30 days' from the
annotation_tombstone_ttl_days tunable (threaded through BookDetail →
NotesHighlightsModal; 1-day singular and retention-disabled variants
handled) — instead of referencing a settings name a reader cannot see.
Objective 8a (web half). Reader notes-drawer rows gain an inline editor
(mirroring the bookmark-rename pattern) that PUTs the new content — the
stored position rides along because the server's note PUT hard-sets the
position column (a probe note proved an omitted position NULLs the
anchor). Book-detail banner rows gain Edit/Delete for location notes
and Edit-note/Add-note for the note ATTACHED to a highlight (full-replace
highlight PUT; empty content clears the note; the highlight itself is
never edited from the banner). The edit dialog is centered with the
markdown caption.
The book-detail 'Notes' button becomes 'Deleted' (badge =
deleted-annotation count) and the modal title becomes 'Deleted
annotations' — that modal only ever showed removed annotations.
Verified live in Brave: drawer inline edit round-trips content and
position; banner note edit, highlight-note edit, and delete (tombstone
appears in the Deleted modal); rename renders with the count badge.
Objective 7 (web half). The 'Annotations' button slides a right-hand
banner (annotations_banner.templ + annotations-banner.ts) fed by the
existing per-book GET endpoints: friendly labels (Section N parsed from
the CFI spine / Page N from the PDF envelope / NN% + first-words
fallbacks), capped highlight previews, rich x-html expansion through the
objective-6 markdown pipeline, and per-row 'Open reader here' jumps
riding /readers/:uuid?loc= (consumeLocJump: jump outranks stored
progress for that open; URL stripped before sync can echo it). WEB-ONLY
export: single .md or one-file-per-item zip via fflate — item format
identical both modes ('# friendly label' / '# selection text' headers,
raw markdown bodies, no raw CFIs anywhere).
Verified live in Brave: 12 restored highlights + a markdown note
labeled/expanded correctly; the jump navigated, landed on the right
chapter and consumed ?loc=; export produced the 13-item .md (friendly
headers only) and a valid 13-entry zip. Book-detail's existing
recently-deleted modal stays untouched (banner is additive).
Objective 6 (web half). Storage/sync stay raw markdown (KOReader shows
literal source, accepted). Notes-grade syntax: headings, emphasis/
strikethrough, code, links, lists, blockquotes, GFM tables, reference
links, footnotes (markdown-it-footnote); images/raw HTML/math excluded
(html:false posture + FORBID img/style/form + default safe-scheme URI
check). Display sites: annotations-drawer note rows and highlight note
lines render pre-sanitized HTML (x-html); render-time sanitize only.
Rich paste: both note textareas intercept paste; a text/html clipboard
flavor converts via turndown (gfm tables incl. a headingless-table rule
emitting pipe syntax; images blanked) and inserts at the caret; plain
pastics fall through to the default paste unchanged.
Verified live in Brave: bold/italic/strike/code/lists/quote/table/
footnote-ref/reference-link render; <script> renders inert literal
text with no alert; javascript: hrefs absent from the DOM; paste of
rich HTML lands markdown at the caret; plain pastes untouched.
One boolean (Security > Public Registration) switches the whole public
signup lifecycle: front page + login links, the logged-out sidebar's
Create-an-account (via a package-level templates hook so shared page
templates keep their signatures), GET /register -> 302 /login, and an
early 403 on POST /api/auth/register. First-user exception preserved:
zero admins keeps every route and link reachable for bootstrapping —
same 'users exist' reasoning as the setup gate. Admin user creation is
unaffected by design.
Live E2E verified: flag off hides all surfaces and blocks the POST
(403), flag on restores them (201); the admin UI renders the row
automatically.
The unqualified 0.0.0.0 publish put Postgres on every interface, and
Docker delivers published ports through PREROUTING DNAT into the
FORWARD path — ufw's default deny incoming never sees those packets.
Result: the database was reachable from whatever network the laptop
joined (home, guest Wi-Fi, hotel), not just from the host.
Bind to 127.0.0.1/[::1] instead: with no DNAT matching LAN-destined
packets, they fall back to INPUT where the firewall actually applies.
Host-side tools keep working over the loopback publish (both families
bound because localhost may resolve to ::1 first); app↔db and tests↔db
are untouched — they use the db service name on the compose network,
which never traverses iptables on this host (br_netfilter not loaded).
If LAN access to the DB is ever wanted again, revert to an unqualified
publish and rely on DOCKER-USER home-subnet scoping instead of an
open binding.
.IMAGE — extensions removed from AllowedExtensions (scan gate),
bookExtensions, MimeTypes and sync/format.go maps: .doc/.lit files
are no longer scanned or indexed. classifyFormatGroup gains the
RTF and PDB reflowable arms that were missing when those rungs
shipped (rows only reclassify on creation). Scanner docs updated.
DOC + LIT have no credible JS tooling (mammoth is docx-only; LIT
needs LZX and its DRM variants are dead) and text-extraction-only
support would misrepresent what the reader can do. Verified: probe
.doc/.lit files watched but never scanned; probe .txt control
scanned; TestClassifyFormatGroup extended and passing.
Docs change granted in-session by the user (extensions + format
docs).
.docx is accepted for ingest but the format-group switch had no case
for it, so every DOCX row stored format_group 'unknown' with
is_reflowable=false. Extract the classification switch into a pure
classifyFormatGroup helper with a table test, and classify .docx
reflowable like the other reflowable text formats (.rtf/.doc stay out
until they are readable client-side). Existing rows: dev alice (DOCX)
reclassified by hand; production rows reclassify on rescan.
Oversized notes previously failed only at save time, surfacing the
raw backend validator error to the reader. The reader now handles
the limit inline and keeps oversized input editable:
- Live character counter under both note textareas (annotations
drawer "Add a note" and the selection popover note editor),
formatted "12,345/100,000" — muted normally, red once over
- Pasting is never truncated; the full text stays in the textarea
so the reader can shrink it however they see fit
- The three save actions (Add Note, Highlight with note, Save
note) disable while over the limit, with a toast fallback in
addNote / createHighlight / saveHighlightChanges so the guard
holds even outside the disabled-button path
- noteMaxLength mirrors the server-side cap (100,000, see
internal/handlers/media.go) so client and server stay linked
Also truncate note rows in the drawer list — highlight rows
already truncated, but a long note previously stretched the
drawer body.
The resize handle needed no work: .reader-note-input already
ships resize: vertical in the compiled CSS; style.css is rebuilt
only for the new disabled:* utility classes.
The 10k validator cap rejected legitimate long-form notes — a
scholarly reading note with quoted passages and footnotes lands
around 10.1k chars and failed at save with the raw validator error.
Raise the cap to 100,000 on all six annotation text fields in the
media handler request structs:
- CreateMediaNoteRequest / UpdateMediaNoteRequest Content
- CreateMediaHighlightRequest / UpdateMediaHighlightRequest NoteText
- CreateMediaBookmarkRequest / UpdateMediaBookmarkRequest Notes
No other layer changes: the media_notes / media_highlights columns
are unbounded TEXT, and the KOReader + websocket sync paths never
had a length cap, so the REST API now matches the rest of the
system instead of being the strictest gate.
New integration test pins the boundary: 50k and 100k-char notes
return 201, 100,001 chars returns 400.
AZW3/KF8 is a proprietary legacy Amazon format (the pipeline moved to
KFX in 2015; KDP dropped MOBI-family uploads in 2022) and does not
render in the web reader. The scanner still ingests .azw/.azw3 files
(code unchanged) — they are simply no longer documented as a supported
expectation.
makeTextBook: UTF-8 decode, blank-line paragraph split, hard-wrapped
lines joined, HTML-escaped, ~150KB section cap at paragraph boundaries
(Part N TOC). Enables the web reader for .txt (previously ingested but
unreadable client-side). Positions for txt books are percentage-based
(sections get index-derived fake CFIs).
Android 16+ gates per-app access to local networks behind a runtime
permission; without it the app's LAN logins time out with zero packets
leaving the phone (browser works — it is not per-app-gated). The new
device guide covers the sideload install, first-run server URL, the
permission prompt (and manual re-grant path), and a triage table for
the login screen's error line. Per the user: overlay-VPN addresses are
not a consideration and stay undocumented.
GetProfile returned the theme from the JWT-claims context stub, which
carries only id/email/username/role — user.Theme was always empty, so
the profile response omitted the field and the app's account-theme
read-back (SERVER sync mode) could never see the stored theme. GetProfile
now loads the full user row by id (h.db.GetUser).
The Android app's app-chrome theme (UX pass item 6) reads the account
theme from GET /api/auth/profile and re-applies it at startup in
SERVER sync mode. PUT /api/auth/theme already persisted the value;
this completes the round trip on the profile read.
Adds publisher_filter to SearchMediaItems/SearchMediaItemsUnified
(fuzzy word_similarity against mi.publisher, mirroring genre_filter),
plumbs it through services.SearchParams and the search handler, and
extends the GREATEST relevance ranking to include publisher matches.
Serves the app's author/publisher/genre/tag click-through browses
(UX pass item 5 — publisher was the only facet without a server
filter).
countTextCharsBefore started its sibling walk at the target node itself
and recursed after counting the current node's subtree, so every
ancestor level re-counted everything accumulated so far — a node at
depth 4 in the single-document 1984 epub reported 2.38M chars before it
in a 589k-char document, producing healed percentages of 4.04 (>1) and
a 500 on the reading_progress percentage check constraint. Every
healed progress save from a fresh client failed; only exact-context
matches (no heal) stored.
The walk now starts at target.PrevSibling: strictly the characters
before the node, per the contract all three callers already assume
(healed book offsets, CRE convert percentage, kepub offsets).
Found by the real-phone validation pass: the phone's first healed
submission 500ed where the emulator's had matched context exactly and
never taken the heal path.
The progress documentation described GET/POST /api/progress/:id —
endpoints that do not exist in the router — while the endpoint every
client actually uses (GET/PUT/DELETE /api/media-items/:id/progress) had
no field-level docs at all.
New:
- progress/position-contract.md: the canonical position model — server
as position authority, the three-tier submission (percentage /
context_text / epubcfi), ingest verification and healing, the restore
handles, the OPF spine numbering hazard (canonical CFI spine steps
include linear="no" items; clients resolve documents by anchor_href
and land by css_selector + char_offset, never by spine step), the
offset currencies (UTF-16 at the wire, runes internal), and
context_text rules.
- progress/get_media_progress.md and update_media_progress.md: the real
endpoints with full field tables, conditionality of the restore
handles, the first-page anti-clobber guard, and healed-response
semantics.
- progress/delete_media_progress.md: the real DELETE route.
- koreader/sync_progress.md and koreader/get_metadata.md rewritten to
the actual payloads: the plugin sends a single-book array whose
"epubcfi" field is a CRE xpointer; the metadata response navigates
via koreader_xpointer (canonical CFI converted back to CRE), with
page as the canonical locator for fixed-layout books.
Removed: the five files documenting the nonexistent /api/progress/:id
GET/POST/DELETE endpoints. Kept get_progress_history.md (that route
exists). The legacy developer/api-reference.md and the indexed
api/api-reference.md progress sections now match the wire and link the
new docs; the duplicate "Universal Progress" section points at Reading
Progress.
Offset currency policy, now explicit: EPUB CFI terminals, CRE text()
offsets and the served char_offset handle are UTF-16 code units (the
EPUB CFI spec, and what foliate/readium/KOReader/Kobo clients actually
observe), while internal arithmetic — the book-wide character_offset
column and percentage fractions — stays rune-based, consistent with
TotalCharacters. For all-BMP books the currencies are identical, so no
stored value changes; astral-plane text (emoji, rare CJK) no longer
drifts.
Boundaries converted: resolveCFIToNode interprets incoming CFI terminal
offsets as UTF-16; textNodeAtUTF16Offset (née textNodeAtRuneOffset)
interprets CRE text() offsets as UTF-16; buildCFI and buildCREXPointer
emit UTF-16 terminals; blockCharOffset (the served char_offset) is
UTF-16.
Also fixes two character_offset column defects: heals wrote a BLOCK-
relative offset into the book-wide column, and verified-but-unhealed
saves (e.g. KOReader pushes) never refreshed it, leaving it stale
behind the anchor. VerifyProgressAnchor now returns the verified book-
wide rune offset and SaveProgress refreshes the column on every
verified save.
Tests: astral currency round trip (offset after an emoji must shift by
one unit between currencies, in both heal and exact-verify directions)
and book-offset ordering. The cmd/server/tests integration harness
failures under docker (library folder 400 during setup) reproduce on
the pre-change tree and are unrelated.
Progress submissions now carry (percentage, context_text, epubcfi) and
the server becomes the position authority:
- VerifyProgressAnchor resolves the submitted standard CFI against the
book's own XHTML, extracts the text at the anchor, and cross-checks it
with the submitted context_text. A mismatch heals the anchor by text
search (percentage disambiguates repeats) instead of storing a bad
position.
- The anchor's block element is derived as a cssSelector plus a block-
relative character offset, and served on progress GET alongside the
anchor document's href — readium-native handles that let clients
re-open a book without parsing CFIs themselves.
- context_text-only submissions (no CFI — the dumb-client tier) are
anchored structurally from the context text.
Motivation: cross-client progress sync (web foliate CFIs, KOReader CRE
xpointers, readium-native apps) previously trusted each client's own
locator math; the app's EPUB restore drifted ±pages because readium's
paginator does not lay out far-from-viewport columns and the foliate-
ported CFI walk ran against readium's mutated WebView DOM. Server-side
verification heals both classes at ingest.
The version tag previously existed only in git and the Docker image tag —
the running app had no way to report what build it was. Version is now
injected at build time via -ldflags into internal/version.Version (defaults
to "dev" for local builds), passed by the release workflow as the
APP_VERSION Docker build arg from the pushed tag.
Surfaced in Admin → Settings → About (extensible card for future rows like
disk space) and in the /health JSON response, so deployments can be
verified with curl alone.
Regenerates all *_templ.go files after the templ generate pass (version
stamp updates across the board; functional changes only in
admin_archived_templ.go and admin_library_templ.go, which were committed
with their features). web/static/style.css and web/static/htmx.min.js are
refreshed outputs of npm run build (tailwind pass over the current
templates; htmx copied from the current node_modules version).
setupTestServer reuses the shared dev admin (testuser@tests.bookhoard.internal)
instead of re-inserting it, but never reset its password — once any test
mutated the admin's password, every later test in the run failed to log in
with 401s until the database was manually wiped. The seeding step now
resets the password hash to the known test constant on reuse, so full
integration runs are repeatable against an existing database.
Adds cross_library_move_test.go, seven integration tests covering the
duplicate-content features end to end:
- TestCrossLibraryMovePreservesHistory: a book moved between two same-type
libraries keeps its row ID — progress and annotations survive, no
duplicate, no archived ghost, library_type_name stays truthful.
- TestCrossLibraryCopyStaysIndependent: deliberate copies in two libraries
stay independent rows with isolated progress.
- TestCrossLibraryMoveRejectedForTypeMismatch: a reflowable EPUB is not
repointed between manga libraries; a library_type_mismatch processing
issue is recorded instead.
- TestListHiddenMediaItemsMatchesActiveTwin: archived rows expose their
active same-SHA twin (and rows without one report no match).
- TestMergeArchivedItemIntoActiveTwin: merging moves progress and
annotations onto the active copy and removes the archived row.
- TestMergeArchivedItemRejections: non-admin 403, active source, hash
mismatch, missing target_id, self-merge, and archived target all fail
closed.
- TestLibraryTypeNameTriggerOnLibraryChange: the UPDATE trigger refreshes
library_type_name when a row changes libraries.
Tests create their own temp-dir library folders (host runs don't have
/app/uploads), use unique device identifiers (device_identifier is
UNIQUE and leftover rows broke reruns), and clean up via an explicit
defer that runs while the pool is still open — a t.Cleanup registered
for the same purpose silently no-ops because it executes after
setup.Close has closed the pool.
The test-runner stage copied only /app from the builder, but the Go module
cache lives in /go/pkg/mod (GOMODCACHE) — the cache mounts used by the
builder target /root/go/pkg/mod, which the go tool ignores, so those mounts
never held anything. Every `compose run tests` / make test-integration
invocation therefore re-downloaded all dependencies from the network,
making integration runs slow and timeout-prone.
The builder now materializes /go/pkg/mod into an image layer
(go-module-cache) after the binary build, and the test-runner stage
restores it to /go/pkg/mod before running tests. Test-only stage; the
production final stage is unaffected.
Three fixes to the Delete Library flow on /admin/library:
1. Honest warning copy. The old text ("Media files will not be deleted")
read as reassurance while deleting a library actually cascade-destroys
every book record with it: reading progress, highlights, notes,
bookmarks, and ratings are gone with no archive window and no undo.
The modal now states this as a scannable list:
- Permanently removed, no archive or undo: the library, its folder
mappings, and all book records — with their reading progress,
highlights, notes, bookmarks, and ratings.
- Not touched: media files on disk.
Admins skim danger dialogs; the irreversible part now leads.
2. The dialog stayed open after confirming. The confirm button swaps
#libraries-container via htmx (so the deleted row vanished) but the
modal lives outside the swapped container and nothing closed it. An
htmx:afterRequest listener now hides the modal on successful deletes
and leaves it open on errors.
3. Latent ReferenceError in openFolderBrowser: the function parameter is
targetInputId but the body referenced an undefined targetInput, so
opening the folder browser threw and the picker never loaded.
The device catalog flattened every visible library into one list, so
duplicate copies of the same book each got an entry and search had no
library context. The root feed is now a standard OPDS 1.1 navigation feed
that mirrors the web UI's library model:
- Root /catalog: an "All Books" entry first (the previous flat cross-
library behavior, also still served at ?all=1 for clients that want
the single flat list), followed by one folder entry per visible
library with live book counts from GetVisibleLibraryMediaCounts.
- New GET /library/:libraryId/catalog: acquisition feed scoped to one
visible library (403 when the device owner cannot see it), paginated,
with an up-link back to the root.
- Scoped search: each library feed's rel="search" OpenSearch template
pins &library_id=<id>, so opening search from inside a library folder
searches only that library — clients substitute only {searchTerms},
so no client-side changes are required. Root search stays global.
- Global search entries now carry the owning library as a category (and
as a fallback summary when the book has no description), so duplicate
copies are distinguishable in unscoped result lists.
The acquisition entry builder is extracted into addAcquisitionEntries and
shared by the all-books and per-library feeds. bookhoard.koplugin needs
no changes: it only registers the root URL, and KOReader's stock OPDS
client renders navigation feeds natively.
Tests: TestOPDSLibraryFolders covers the nav-feed shape, flat ?all=1
mode, scoped catalog isolation, scoped/global search behavior, and 403s
for libraries outside the device's visibility. Library names avoid the
word "test" on purpose — setupDeviceTest re-runs setupTestServer's
setup-time cleanup, which deletes every library whose name contains it.
Covers the copy-then-delete-later workflow: a user copies books to a new
library, deletes the originals, and ends up with an active copy in the new
library plus an archived twin holding the real reading history. Until now
those twins could only be purged (destroying the history) or restored
(showing a permanently broken entry).
- POST /api/media-items/:id/merge (admin only, body {target_id}):
validates the source is archived/missing, the target is active, and
both share the same file_sha256; then re-parents every child row onto
the target via the existing reparent_media_item_children function (the
same machinery as hash-conflict resolution) and deletes the source row.
Per-user collisions keep the active copy's data, mirroring that flow.
Affected data: reading progress, speed, ratings, highlights, notes,
bookmarks (including tombstoned deleted-annotation history), formats,
collections, kobo shelves/entitlements, sync rows, panel data,
processing issues, and device aliases.
- GET /admin/archived: ListHiddenMediaItems' match_* columns surface each
row's best active twin; rows with a twin get a confirm-guarded "Merge"
button (data-merge-source/-target) next to Restore/Delete, wired in
web/src/admin.ts like the existing unarchive/delete handlers.
- templates.ArchivedItem gains MatchID/MatchTitle/MatchLibraryName;
frontend.go populates them from the listing row.
After a merge the archived row is gone, so the retention purge can never
destroy the merged data. Deleted-annotation tombstones carry over and
remain restorable from the target book's "recently deleted" history.
Previously the scanner's SHA-256 dedup was library-scoped: moving a book
between libraries created a duplicate row (new ID) while the old row went
missing and archived after two scans, orphaning reading progress and
annotations from the file that users still see.
processMediaFile now falls through to cross-library detection when the
same-library hash lookup misses:
- GetMediaItemsBySHA256AndSameLibraryType returns candidates in other
same-type libraries; each candidate's file is stat'd through ITS OWN
library's folders (the old existence check stat'd against the current
scan's folder tree, which is meaningless across libraries).
- File gone -> it is a move: the candidate must pass the target library's
type rules (ValidateMediaItemForLibrary), then the row is repointed via
MoveMediaItemToLibrary with the recomputed relative path. Reading
history, annotations, and collections follow automatically because the
row keeps its ID. Incompatible formats (e.g. a reflowable EPUB into a
manga library) are rejected with a processing issue instead of being
force-imported; the upsert on (media_item_id, issue_type) keeps
repeated scans from spamming duplicates.
- File still present -> deliberate multi-library copy: fall through to
normal import so both libraries keep independent rows.
Candidate selection is factored into selectMoveCandidate (pure function,
unit-tested in media_scanner_move_test.go): input arrives pre-ordered
(archived first, then most missing scans, then oldest) and only
candidates whose file is verifiably gone qualify, so deliberate copies
are never repointed.
Three related query groups in queries.sql (sqlc regenerated; no generated
signatures changed, so no caller edits were needed):
1. Dashboard/collection archive filters. The smart-section and collection
queries returned archived and missing items (visible only as broken
covers once their files vanished). Added the codebase-standard
"AND archived_at IS NULL AND missing_scan_count = 0" predicate to:
GetContinueReadingItems, GetRecentlyAddedItems, GetRecentlyReadItems,
GetNotStartedItems, GetCollectionItemsForDashboard, GetLibraryItems.
Matches the existing convention in ListMediaItems, SearchMediaItems,
the library counts, and GetContinueSeriesItems.
2. Cross-library SHA move detection support.
- GetMediaItemsBySHA256AndSameLibraryType: finds identical content
(same file_sha256) registered in a DIFFERENT library of the SAME
library type, ordered archived-first, most-missing, oldest. Type
scoping keeps ebooks from merging into manga/comics libraries.
- MoveMediaItemToLibrary: repoints an existing row to the new library
(library_id + recomputed file_path + file_size) and clears
archive/missing state. The row keeps its ID, so reading progress,
annotations, collections, and kobo shelves follow the book
automatically; the new set_library_type_name_on_library_change
trigger refreshes the denormalized type column.
3. Archived-twin matching. ListHiddenMediaItems (admin archived-items
page) now LEFT JOIN LATERALs the best active same-SHA twin per hidden
row — preferring same library type, then same library, then most
reading progress, then oldest — exposing match_id / match_title /
match_library_name so the UI can offer merging the archived row's
reading data into its active duplicate. COALESCE on match_title keeps
the no-match case NULL-safe.
schema.sql is re-executed in full on every application startup, so every
statement in it must be idempotent. Two related problems introduced with the
cross-library move feature, plus their fix:
Problem 1: media_items.library_type_name was only populated by a BEFORE
INSERT trigger. When the scanner repoints an existing row to a different
library (cross-library move detection), the denormalized library_type_name
went stale, mislabeling the item's type for validation and display.
Fix: add a second trigger, set_library_type_name_on_library_change, that
fires BEFORE UPDATE OF library_id and re-runs the same population function.
Problem 2 (outage): the new trigger EXECUTE FUNCTIONs set_library_type_name(),
but the pre-existing idempotency dance drops that function on every startup
before recreating it. On the second and later boots, DROP FUNCTION failed
with SQLSTATE 2BP01 (function still depended on by the trigger created by
the previous boot), aborting the whole schema transaction and crash-looping
the container.
Fix: drop BOTH triggers before dropping the function, and create both after
it. The ordering now survives any number of restarts on any database state.
Adds TestSchemaInitializationIsIdempotent, which replays the production
database.Initialize twice against the same database so this class of
"second startup" regression fails in CI instead of in production. Note the
test uses a dedicated pgxpool: Initialize holds the advisory lock on one
connection while executing the schema on another, which deadlocks against
the shared test pool's MaxConns=1.
go mod download all added sums for modules only reachable from test
builds (chroma, bluemonday, compress families); without them the
integration-test binary build fetches missing sums at compile time.
Two pre-existing harness breaks found while running the device tests:
- RegisterRoutes dereferences cfg.Settings (auth rate limit) but
setupTestServer never set it — every integration test nil-panicked at
route registration. Wire database.NewSettingsRegistry(queries) the
same way main.go does.
- The cleanup deliberately PRESERVES testuser@tests.bookhoard.internal
(shared dev admin), but setup then blindly re-INSERTed that user, so
every run after the first failed on users_email_key. Reuse the user
when it exists; default collections are created only for a NEW user
(the preserved admin already has its set).
Also documented the offline runner used on slow links: build the test
binary on the host (CGO_ENABLED=0 go test -c) and execute it inside the
bookhoard-tests image against the compose network — no in-container
module downloads, no image rebuild.
The mutex hardening added a fall-through 'pending' response path that
returned while still holding pendingMu — one status poll before approval
leaked the lock and every later register/approve/status request hung
forever (caught by TestDeviceRegistrationFlow's approve step hanging).
App reinstalls that preserve data (Android Studio installDebug over an
existing install) re-register with the same device_identifier, but the
devices row from the previous install still exists — device_identifier
is UNIQUE, so ApproveDevice's blind INSERT failed with a unique
violation and returned 500 'failed to create device' (reproduced via
curl: second approve with the same identifier = instant 500; the ~98s
in the original report was app-side retry/polling, not server wait).
ApproveDevice is now idempotent: look the device up by identifier
first; a row owned by the approving user gets its auth token rotated
via UpdateDeviceAuthToken (row id unchanged, so synced highlights/
bookmarks/progress anchored to it stay valid; fresh install = fresh
credentials, old token invalidated); a row owned by another user gets
409; unknown identifiers INSERT as before, with the 23505 race falling
through to the rotate path. DB failures are logged (they were silent).
Also guard the in-memory pendingRegistrations map with a mutex —
register/approve/reject/status/list all touch it from HTTP goroutines,
and a racing write is a Go runtime fatal, not an error. The approver's
credential publication and the status poller's approved-branch snapshot
now run under the lock so the token can never be read half-written.
Regression test: TestApproveDeviceReapprovalRotatesToken — register →
approve → re-register same identifier → approve (must be 200) → token
rotated, exactly one devices row, row carries the new token.
User-facing documentation for the rebuilt web reader (§6.5 of the app
handoff): swipe-to-page / tap-for-menu touch navigation, long-press word
selection with drag extension and handles, the selection popover (color
dots, notes with save-together-on-create, copy incl. plain-HTTP LAN,
edit/delete by tapping a painted highlight), below-the-selection popover
placement rationale, PDF/comic behavior, and annotations drawer.
Linked from the user documentation portal and the docs index quick
links / quick-find tables.
The lockfile is gitignored (each machine keeps its own), but
.dockerignore did not exclude it, so any stale local lock rode into
every docker build via `COPY package*.json`. Because the forked
foliate-js declares "version": "0.0.0" on every commit, npm treats
the git pin as already satisfied by name@version and never
re-resolves the new commit hash — silently installing and bundling
the old code. This bit both the host npm cache mount (documented at
Dockerfile:18-20) and, today, `make rebuild-app-force`: a fresh
no-cache image was built with the pre-feature 1305a52 foliate-js
(chunk fixed-layout-B8-qRQLl.js) despite package.json pinning
e16530a, while the Gitea runner (fresh checkout, no lockfile, cold
cache) built correctly.
With no lockfile in the context, npm install resolves git pins
fresh from package.json each build (tarballs are cached by
commit-specific URLs), so the persistent npm cache mount cannot
serve old commits across pin bumps. Local lockfiles can no longer
poison builds even if regenerated on the host.
Verified: after evicting the poisoned cache mounts
(docker builder prune --filter type=exec.cachemount) and rebuilding,
the container serves fixed-layout-BE0KdOql.js with both dblclick
handlers present, matching the reference build.
2026-09-17 15:37:58 -04:00
384 changed files with 32957 additions and 15830 deletions
- ❌ **NEVER duplicate business logic** - keep logic in services, not handlers
- ❌ **NEVER bypass service layer** - all database operations must go through services
### Testing
- ✅ **ALWAYS use `setupTestServer()` helper from `cmd/server/tests/test_helpers_test.go`*
- ✅ **Share one test setup across all subtests** - call `setupTestServer()` once at test function level, not per subtest
- ✅ **Prefer table-driven tests** - use `t.Run()` with test cases instead of duplicate test functions
- ✅ **Configure database pools efficiently** - use `max_conns=1` for test pools (via `pgxpool.ParseConfig()`) to prevent connection exhaustion
- ❌ **NEVER create separate `pgxpool` per test** - each pool creates 4 connections by default; 78 tests = 312 potential connections > PostgreSQL's 100 limit
- ❌ **NEVER call `setupTestServer()` in loops or within subtests** - creates unnecessary database pools and exhausts connections
- ✅ **DO verify tests pass** - run full test suite before completing work
- ✅ **Use `t.Cleanup()` properly** - the `TestServerSetup` pattern automatically handles cleanup via `t.Cleanup()`
### Frontend & Styling
- ❌ **NEVER modify backend/API for frontend features without user confirmation*
- ❌ **NEVER use custom CSS** - TailwindCSS classes only
- **⚠️ EXCEPTION**: `templates/error.templ` may have inline CSS because error pages must work when main app fails (404, server errors, CSS fails to load)
- ❌ **NEVER use JavaScript** - convert all to TypeScript
- **⚠️ EXCEPTION**: Inline JS function calls in HTML attributes (e.g., `onclick="myFunction()"`) are acceptable for simple interactions
- ❌ **NEVER use Object-Oriented Programming** (no classes, inheritance, or this-capture)
- ✅ **DO use procedural/imperative style** as your default
- ✅ **DO borrow functional techniques** when they simplify code
- ✅ **DO avoid ideological purity** - the best paradigm is the one that fits the problem
- ❌ **NEVER add new Dockerfiles without user confirmation*
- ❌ **NEVER fetch initial data via AJAX on page load** - use server-side rendering instead
- ❌ **NEVER break progressive enhancement** - pages must work without JavaScript
**Note:** Go methods in the backend are fine and encouraged. This guideline applies to TypeScript/JavaScript frontend code only.
- ❌ **NEVER commit files with secrets** (.env, credentials.json, etc.)
- ❌ **NEVER make assumptions** - ask clarifying questions when uncertain
- ❌ **NEVER delete code without reading full context first** (minimum 20 lines before/after)
- ❌ **NEVER make cascading fix-up edits without git diff review*
- After compilation error: STOP, review `git diff`, understand full impact
- Use revert/reapply pattern instead of blind fixes
- ❌ **NEVER skip post-edit verification** - must compile after each file edit
- ❌ **NEVER run git commands concurrently** - always run sequentially:
- Run `git add <files>` and wait for completion
- Run `git commit -m "<message>"` and wait for completion
- Run `git push` and wait for completion
- Never use `&&` to chain git commands together
- ✅ **Make good organized git commits for the entire project (not just what you changed) and push - run git add, commit, push sequentially as separate commands, no need to repeatedly check status*
- ✅ **Indentation is always 2 spaces unless the language prohibits it*
### Cascading Fix-up Pattern (PROHIBITED)
**WHAT NOT TO DO** - This caused critical bugs:
```go
// ❌ WRONG: Blindly making fixes after compilation error
Edit1:Deletedeprecatedcode
[Compilationerror:undefinedRegister]
Edit2:Trytofixerror(over-broaddeletion)
[Moreerrors:undefinedLogin,GetProfile,etc.]
Edit3:Trytofixagain(worsedamage)
[Evenmoreerrors:majorfunctionalitybroken]
```
**CORRECT APPROACH**:
```go
// ✅ CORRECT: Stop, understand, then fix deliberately
- `media_item_ids` (array of strings): Array of media item UUIDs to delete
**Response:**
- `results` (array): Results for each deletion attempt
- `total` (number): Total number of media items processed
- `deleted` (number): Number of successfully deleted media items
- `failed` (number): Number of failed deletions
**Status Codes:**
- 200: Success (with partial results if some failed)
- 400: Invalid request data
- 401: Unauthorized
- 403: Forbidden
- 500: Internal server error
**Example:**
```json
{
"media_item_ids": [
"uuid-1",
"uuid-2",
"uuid-3"
]
docs:"## Bulk Delete Media Items\n\n Deletes multiple media items in a single request.\n\n **Method:** POST\n\n **Endpoint:** /api/media-items/bulk-delete\n\n **Authentication:** Required (Bearer token)\n\n **Request Body:**\n - `media_item_ids` (array of strings): Array of media item UUIDs to delete\n\n **Response:**\n - `results` (array): Results for each deletion attempt\n - `total` (number): Total number of media items processed\n - `deleted` (number): Number of successfully deleted media items\n - `failed` (number): Number of failed deletions\n\n **Status Codes:**\n - 200: Success (with partial results if some failed)\n - 400: Invalid request data\n - 401: Unauthorized\n - 403: Forbidden\n - 500: Internal server error\n\n **Example:**\n ```json\n {\n \"media_item_ids\": [\n \"uuid-1\",\n \"uuid-2\",\n \"uuid-3\"\n ]"
Updates multiple media items in a single request with different fields for each item.
**Method:** POST
**Endpoint:** /api/media-items/bulk-update
**Authentication:** Required (Bearer token)
**Request Body:**
- `media_item_updates` (array): Array of update objects
- `media_item_id` (string): Media item UUID to update
- `updates` (object): Fields to update (can include title, author, genre, tags, contributors, etc.)
**Response:**
- `results` (array): Results for each update attempt
- `total` (number): Total number of media items processed
- `updated` (number): Number of successfully updated media items
- `failed` (number): Number of failed updates
**Status Codes:**
- 200: Success (with partial results if some failed)
- 400: Invalid request data
- 401: Unauthorized
- 403: Forbidden
- 500: Internal server error
**Example:**
```json
{
"media_item_updates": [
{
"media_item_id": "uuid-1",
"updates": {
"title": "New Title",
"genre": "Fiction",
"tags": ["fiction", "adventure"]
docs:"## Bulk Update Media Items\n\n Updates multiple media items in a single request with different fields for each item.\n\n **Method:** POST\n\n **Endpoint:** /api/media-items/bulk-update\n\n **Authentication:** Required (Bearer token)\n\n **Request Body:**\n - `media_item_updates` (array): Array of update objects\n - `media_item_id` (string): Media item UUID to update\n - `updates` (object): Fields to update (can include title, author, genre, tags, contributors, etc.)\n\n **Response:**\n - `results` (array): Results for each update attempt\n - `total` (number): Total number of media items processed\n - `updated` (number): Number of successfully updated media items\n - `failed` (number): Number of failed updates\n\n **Status Codes:**\n - 200: Success (with partial results if some failed)\n - 400: Invalid request data\n - 401: Unauthorized\n - 403: Forbidden\n - 500: Internal server error\n\n **Example:**\n ```json\n {\n \"media_item_updates\": [\n {\n\
docs:"Create a custom collection with filter rules.\n\n**Endpoint**: POST /api/collections\n**Auth**: Required (Bearer token)\n\n## Request Body\n\n| Field | Type | Required | Description |\n|-------|------|----------|-------------|\n| library_id | string | Yes | Library UUID |\n| name | string | Yes | Collection name |\n| icon | string | No | Emoji icon |\n| description | string | No | Collection description |\n| show_on_dashboard | boolean | No | Show on dashboard (default: true) |\n| auto_assign_rules | string | No | JSON string of filter rules |\n| manual_book_ids | array | No | Array of book IDs |\n| match_type | string | No | \"all\" (AND) or \"any\" (OR) |\n\n## Example Request\n\n```json\n{\n \"library_id\": \"cc23c3a7-f8fb-451a-a78d-2a16df1b725a\",\n \"name\": \"Science Fiction Books\",\n \"icon\": \"\U0001F680\",\n \"description\": \"All my sci-fi books\",\n \"show_on_dashboard\": true,\n \"auto_assign_rules\": \"{\\\"rules\\\":[{\\\"id\\\":\\\"rule1\\\",\\\"field\\\":\\\
\"genre\\\",\\\"operator\\\":\\\"equals\\\",\\\"value\\\":\\\"Sci-Fi\\\",\\\"priority\\\":1}]}\",\n \"manual_book_ids\": [],\n \"match_type\": \"all\"\n}\n```\n\n## Response\n\nReturns the created collection with generated ID."
This endpoint is used by the Custom Section Builder to show users what books will match their filter rules before they save the collection.
- type:after-response
code:"function onResponse(res) {\ntest(\"status must be 200 with valid request\", function() {\n expect(res.status).to.eql(200);\n});\n\ntest(\"response contains items array\", function() {\n expect(res.body.items).to.exist;\nexpect(res.body.items).to.be.an('array');\n });\n}"
docs:"Preview collection with filter rules before saving.\n\n**Endpoint**: POST /api/collections/preview\n**Auth**: Required (Bearer token)\n\n## Request Body\n\n| Field | Type | Required | Description |\n|-------|------|----------|-------------|\n| library_id | string | Yes | Library UUID |\n| rules | array | No | Filter rules to evaluate |\n| manual_book_ids | array | No | Manually selected book IDs |\n| limit | int | No | Max items to return (default: 20, max: 100) |\n\n## Example Request\n\n```json\n{\n \"library_id\": \"cc23c3a7-f8fb-451a-a78d-2a16df1b725a\",\n \"rules\": [\n {\n \"id\": \"rule1\",\n \"field\": \"genre\",\n \"operator\": \"equals\",\n \"value\": \"Fiction\",\n \"priority\": 1\n }\n ],\n \"manual_book_ids\": [],\n \"limit\": 20\n}\n```\n\n## Response\n\nReturns array of matching book items.\n\nThis endpoint is used by the Custom Section Builder to show users what books will match their filter rules before they save the collection."
Adds multiple books to multiple collections in a single request. Each operation specifies a collection and a list of books to add.
**Method:** POST
**Endpoint:** /api/collections/bulk-add-books
**Authentication:** Bearer token
**Request Body:**
- `operations` (array): Array of collection-book operations
- `collection_id` (string): Collection UUID
- `book_ids` (array): Array of book UUIDs to add to the collection
**Response:**
- `results` (array): Results for each operation
- `total` (number): Total number of operations
- `success` (number): Number of successful operations
- `failed` (number): Number of failed operations
**Status Codes:**
- 200: Success (with partial results if some failed)
- 400: Invalid request data
- 401: Unauthorized
- 403: Forbidden
- 500: Internal server error
**Example:**
```json
{
"operations": [
{
"collection_id": "collection-uuid-1",
"book_ids": ["book-1", "book-2"]
docs:"## Bulk Add Books to Collections\n\n Adds multiple books to multiple collections in a single request. Each operation specifies a collection and a list of books to add.\n\n **Method:** POST\n\n **Endpoint:** /api/collections/bulk-add-books\n\n **Authentication:** Bearer token\n\n **Request Body:**\n - `operations` (array): Array of collection-book operations\n - `collection_id` (string): Collection UUID\n - `book_ids` (array): Array of book UUIDs to add to the collection\n\n **Response:**\n - `results` (array): Results for each operation\n - `total` (number): Total number of operations\n - `success` (number): Number of successful operations\n - `failed` (number): Number of failed operations\n\n **Status Codes:**\n - 200: Success (with partial results if some failed)\n - 400: Invalid request data\n - 401: Unauthorized\n - 403: Forbidden\n - 500: Internal server error\n\n **Example:**\n ```json\n {\n \"operations\": [\n {\n \"collection_id\":\
Removes multiple books from a collection in a single request.
**Method:** POST
**Endpoint:** /api/collections/{collection_id
docs:"## Bulk Remove Books from Collection\n\n Removes multiple books from a collection in a single request.\n\n **Method:** POST\n\n **Endpoint:** /api/collections/{collection_id"
Tests validation behavior when providing an empty book_ids array.
**Expected Result:** 400 Bad Request
**Validation Rule:** book_ids array must contain at least one book UUID.
**Purpose:** Ensures the API properly validates input and rejects empty removal requests.
docs: "## Bulk Remove Books - Empty List Validation\n\n Tests validation behavior when providing an empty book_ids array.\n\n **Expected Result:** 400 Bad Request\n\n **Validation Rule:** book_ids array must contain at least one book UUID.\n\n **Purpose:** Ensures the API properly validates input and rejects empty removal requests."
Tests behavior when the book_ids array contains invalid UUID formats or non-existent books.
**Expected Result:** 200 OK with partial success
**Purpose:** Verifies that:
- Invalid UUID formats don't crash the endpoint
- Non-existent book IDs are handled gracefully
- Valid IDs in the same request are still processed
- Response includes detailed results showing which succeeded/failed
**Note:** The endpoint should process all valid IDs and report failures for invalid ones, allowing clients to handle partial failures appropriately.
docs:"## Bulk Remove Books - Invalid IDs\n\n Tests behavior when the book_ids array contains invalid UUID formats or non-existent books.\n\n **Expected Result:** 200 OK with partial success\n\n **Purpose:** Verifies that:\n - Invalid UUID formats don't crash the endpoint\n - Non-existent book IDs are handled gracefully\n - Valid IDs in the same request are still processed\n - Response includes detailed results showing which succeeded/failed\n\n **Note:** The endpoint should process all valid IDs and report failures for invalid ones, allowing clients to handle partial failures appropriately."
Tests that bulk remove endpoint works correctly with a single book.
**Expected Result:** 200 OK with removed: 1, total: 1
**Purpose:** Verifies the bulk remove endpoint handles single-item arrays correctly, providing flexibility for clients to use the same endpoint for both single and multiple removals.
**Note:** Using bulk remove for a single book is functionally equivalent to the single remove endpoint but allows for consistent error handling and response format.
docs:"## Bulk Remove Books - Single Book\n\n Tests that bulk remove endpoint works correctly with a single book.\n\n **Expected Result:** 200 OK with removed: 1, total: 1\n\n **Purpose:** Verifies the bulk remove endpoint handles single-item arrays correctly, providing flexibility for clients to use the same endpoint for both single and multiple removals.\n\n **Note:** Using bulk remove for a single book is functionally equivalent to the single remove endpoint but allows for consistent error handling and response format."
Tests the "contains" operator on the author field to find books by a specific author (partial match).
**Example Use Case:** Finding all books by an author whose name contains "Asimov" (e.g., "Isaac Asimov").
**Operator:** `contains` - Matches if the field contains the specified value as a substring (case-insensitive typically).
**Expected Result:** Returns all books where the author field contains "Asimov".
**Purpose:** Demonstrates text-based partial matching for author searches, useful when you don't need the exact author name or want to find books by authors with similar names.
docs:"## Test Collection Rules - Author Contains\n\n Tests the \"contains\" operator on the author field to find books by a specific author (partial match).\n\n **Example Use Case:** Finding all books by an author whose name contains \"Asimov\" (e.g., \"Isaac Asimov\").\n\n **Operator:** `contains` - Matches if the field contains the specified value as a substring (case-insensitive typically).\n\n **Expected Result:** Returns all books where the author field contains \"Asimov\".\n\n **Purpose:** Demonstrates text-based partial matching for author searches, useful when you don't need the exact author name or want to find books by authors with similar names."
## Test Collection Rules - Copyright Year Greater Than
Tests the "greater_than" operator on the copyright_year field to find books published after a specific year.
**Example Use Case:** Creating a "Modern Books" collection with books published after 2000.
**Operator:** `greater_than` - Matches if the field value is greater than the specified value (numeric comparison).
**Field:** `copyright_year` - The year the book was copyrighted/published.
**Expected Result:** Returns all books with copyright_year greater than 2000 (i.e., published in 2001 or later).
**Purpose:** Demonstrates numeric comparison operators for creating date-based collections, useful for organizing books by publication era.
docs:"## Test Collection Rules - Copyright Year Greater Than\n\n Tests the \"greater_than\" operator on the copyright_year field to find books published after a specific year.\n\n **Example Use Case:** Creating a \"Modern Books\" collection with books published after 2000.\n\n **Operator:** `greater_than` - Matches if the field value is greater than the specified value (numeric comparison).\n\n **Field:** `copyright_year` - The year the book was copyrighted/published.\n\n **Expected Result:** Returns all books with copyright_year greater than 2000 (i.e., published in 2001 or later).\n\n **Purpose:** Demonstrates numeric comparison operators for creating date-based collections, useful for organizing books by publication era."
Tests validation behavior when providing an empty rules array.
**Expected Result:** 400 Bad Request
**Validation Rule:** rules array must contain at least one rule object.
**Purpose:** Ensures the API properly validates input and rejects empty rule sets, preventing accidental queries that would return all books or cause performance issues.
**Use Case:** Client-side validation should prevent sending empty rules, but the API should also validate to catch malformed requests.
docs: "## Test Collection Rules - Empty Rules Array\n\n Tests validation behavior when providing an empty rules array.\n\n **Expected Result:** 400 Bad Request\n\n **Validation Rule:** rules array must contain at least one rule object.\n\n **Purpose:** Ensures the API properly validates input and rejects empty rule sets, preventing accidental queries that would return all books or cause performance issues.\n\n **Use Case:** Client-side validation should prevent sending empty rules, but the API should also validate to catch malformed requests."
Tests behavior when collection rules don't match any books in the library.
**Example Use Case:** Validating that a new genre name doesn't exist before creating a collection for it, or testing edge cases.
**Expected Result:** 200 OK with empty matches array and total: 0
**Purpose:** Verifies that the API handles zero-match scenarios gracefully:
- Returns 200 (success) not 404
- Returns empty array, not null
- Returns total: 0 for clarity
- No errors thrown for no results
**Note:** An empty result set is a valid response and doesn't indicate an error. This allows users to test rules confidently before creating collections.
docs:"## Test Collection Rules - No Matches\n\n Tests behavior when collection rules don't match any books in the library.\n\n **Example Use Case:** Validating that a new genre name doesn't exist before creating a collection for it, or testing edge cases.\n\n **Expected Result:** 200 OK with empty matches array and total: 0\n\n **Purpose:** Verifies that the API handles zero-match scenarios gracefully:\n - Returns 200 (success) not 404\n - Returns empty array, not null\n - Returns total: 0 for clarity\n - No errors thrown for no results\n\n **Note:** An empty result set is a valid response and doesn't indicate an error. This allows users to test rules confidently before creating collections."
Tests collection rules against the library to see which books match, without creating a collection. Useful for previewing what books would be included in a collection with specific rules.
**Method:** POST
**Endpoint:** /api/collections/test-rules
**Authentication:** Bearer token
**Request Body:**
- `rules` (array): Array of rule objects to test
- `field` (string): Field to test (genre, author, copyright_year, tags, etc.)
- `operator` (string): Comparison operator
- `equals`: Exact match
- `contains`: Contains substring (for text fields)
- `greater_than`: Greater than (for numeric fields)
- `less_than`: Less than (for numeric fields)
- `not_equals`: Not equal to
- `starts_with`: Starts with
- `ends_with`: Ends with
- `is_empty`: Field is empty or null
- `is_not_empty`: Field is not empty and not null
- `value` (string): Value to compare against (not required for is_empty/is_not_empty)
**Response:**
- `matches` (array): Array of matching books
- `id` (string): Book UUID
- `title` (string): Book title
- `author` (string): Book author
- `genre` (string): Book genre
- Additional book metadata
- `total` (number): Total number of matching books
docs:"## Test Collection Rules\n\n Tests collection rules against the library to see which books match, without creating a collection. Useful for previewing what books would be included in a collection with specific rules.\n\n **Method:** POST\n\n **Endpoint:** /api/collections/test-rules\n\n **Authentication:** Bearer token\n\n **Request Body:**\n - `rules` (array): Array of rule objects to test\n - `field` (string): Field to test (genre, author, copyright_year, tags, etc.)\n - `operator` (string): Comparison operator\n - `equals`: Exact match\n - `contains`: Contains substring (for text fields)\n - `greater_than`: Greater than (for numeric fields)\n - `less_than`: Less than (for numeric fields)\n - `not_equals`: Not equal to\n - `starts_with`: Starts with\n - `ends_with`: Ends with\n - `is_empty`: Field is empty or null\n - `is_not_empty`: Field is not empty and not null\n - `value` (string): Value to compare against (not required\
\ for is_empty/is_not_empty)\n\n **Response:**\n - `matches` (array): Array of matching books\n - `id` (string): Book UUID\n - `title` (string): Book title\n - `author` (string): Book author\n - `genre` (string): Book genre\n - Additional book metadata\n - `total` (number): Total number of matching books\n\n **Status Codes:**\n - 200: Success - returns matching books\n - 400: Invalid request (empty rules array, invalid field/operator)\n - 401: Unauthorized\n - 500: Internal server error\n\n **Example Request:**\n ```json\n {\n \"rules\": [\n {\n \"field\": \"genre\",\n \"operator\": \"equals\",\n \"value\": \"Science Fiction\""
Resolves a sync conflict by choosing which source to use for the conflicting data.
**Method:** POST
**Endpoint:** /api/conflicts/{conflict_id
docs:"## Resolve Conflict\n\n Resolves a sync conflict by choosing which source to use for the conflicting data.\n\n **Method:** POST\n\n **Endpoint:** /api/conflicts/{conflict_id"
Dismisses multiple sync conflicts without resolving them. This removes them from the conflict list while leaving the data unchanged.
**Method:** POST
**Endpoint:** /api/conflicts/bulk-dismiss
**Authentication:** Bearer token
**Request Body:**
- `conflict_ids` (array): Array of conflict UUIDs to dismiss
**Response:**
- `results` (array): Results for each dismissal
- `total` (number): Total number of conflicts processed
- `success` (number): Number of successfully dismissed conflicts
- `failed` (number): Number of failed dismissals
**Status Codes:**
- 200: Success
- 400: Invalid request data
- 401: Unauthorized
- 403: Forbidden
- 404: One or more conflicts not found
- 500: Internal server error
**Example:**
```json
{
"conflict_ids": ["uuid-1", "uuid-2"]
docs:"## Bulk Dismiss Conflicts\n\n Dismisses multiple sync conflicts without resolving them. This removes them from the conflict list while leaving the data unchanged.\n\n **Method:** POST\n\n **Endpoint:** /api/conflicts/bulk-dismiss\n\n **Authentication:** Bearer token\n\n **Request Body:**\n - `conflict_ids` (array): Array of conflict UUIDs to dismiss\n\n **Response:**\n - `results` (array): Results for each dismissal\n - `total` (number): Total number of conflicts processed\n - `success` (number): Number of successfully dismissed conflicts\n - `failed` (number): Number of failed dismissals\n\n **Status Codes:**\n - 200: Success\n - 400: Invalid request data\n - 401: Unauthorized\n - 403: Forbidden\n - 404: One or more conflicts not found\n - 500: Internal server error\n\n **Example:**\n ```json\n {\n \"conflict_ids\": [\"uuid-1\", \"uuid-2\"]"
Resolves multiple sync conflicts in a single request using a specified resolution strategy.
**Method:** POST
**Endpoint:** /api/conflicts/bulk-resolve
**Authentication:** Bearer token
**Request Body:**
- `conflict_ids` (array): Array of conflict UUIDs to resolve
- `strategy` (string): Resolution strategy
- `most_recent`: Use the most recently updated progress
- `highest_progress`: Use the reading progress with the highest percent read
- `server`: Always prefer server-side data
- `device`: Always prefer device-side data
**Response:**
- `results` (array): Results for each conflict resolution
- `total` (number): Total number of conflicts processed
- `success` (number): Number of successfully resolved conflicts
- `failed` (number): Number of failed resolutions
**Status Codes:**
- 200: Success (with partial results if some failed)
- 400: Invalid request data
- 401: Unauthorized
- 403: Forbidden
- 404: One or more conflicts not found
- 500: Internal server error
**Example:**
```json
{
"conflict_ids": ["uuid-1", "uuid-2", "uuid-3"],
"strategy": "most_recent"
docs:"## Bulk Resolve Conflicts\n\n Resolves multiple sync conflicts in a single request using a specified resolution strategy.\n\n **Method:** POST\n\n **Endpoint:** /api/conflicts/bulk-resolve\n\n **Authentication:** Bearer token\n\n **Request Body:**\n - `conflict_ids` (array): Array of conflict UUIDs to resolve\n - `strategy` (string): Resolution strategy\n - `most_recent`: Use the most recently updated progress\n - `highest_progress`: Use the reading progress with the highest percent read\n - `server`: Always prefer server-side data\n - `device`: Always prefer device-side data\n\n **Response:**\n - `results` (array): Results for each conflict resolution\n - `total` (number): Total number of conflicts processed\n - `success` (number): Number of successfully resolved conflicts\n - `failed` (number): Number of failed resolutions\n\n **Status Codes:**\n - 200: Success (with partial results if some failed)\n - 400: Invalid request data\n - 401: Unauthorized\n\
\ - 403: Forbidden\n - 404: One or more conflicts not found\n - 500: Internal server error\n\n **Example:**\n ```json\n {\n \"conflict_ids\": [\"uuid-1\", \"uuid-2\", \"uuid-3\"],\n \"strategy\": \"most_recent\""
Resolves multiple sync conflicts using the "highest_progress" strategy, which keeps the reading progress with the highest percentage read.
**Method:** POST
**Endpoint:** /api/conflicts/bulk-resolve
**Authentication:** Bearer token
**Request Body:**
- `conflict_ids` (array): Array of conflict UUIDs to resolve
- `strategy` (string): Must be "highest_progress"
**Response:**
- `results` (array): Results for each conflict resolution
- `total` (number): Total number of conflicts processed
- `success` (number): Number of successfully resolved conflicts
- `failed` (number): Number of failed resolutions
**Status Codes:**
- 200: Success
- 400: Invalid request data
- 401: Unauthorized
- 403: Forbidden
- 500: Internal server error
**Note:** The highest progress strategy is ideal when you want to preserve the most reading progress across devices. Use this when you've been reading on multiple devices and want to keep the furthest position.
docs:"## Bulk Resolve with Highest Progress Strategy\n\n Resolves multiple sync conflicts using the \"highest_progress\" strategy, which keeps the reading progress with the highest percentage read.\n\n **Method:** POST\n\n **Endpoint:** /api/conflicts/bulk-resolve\n\n **Authentication:** Bearer token\n\n **Request Body:**\n - `conflict_ids` (array): Array of conflict UUIDs to resolve\n - `strategy` (string): Must be \"highest_progress\"\n\n **Response:**\n - `results` (array): Results for each conflict resolution\n - `total` (number): Total number of conflicts processed\n - `success` (number): Number of successfully resolved conflicts\n - `failed` (number): Number of failed resolutions\n\n **Status Codes:**\n - 200: Success\n - 400: Invalid request data\n - 401: Unauthorized\n - 403: Forbidden\n - 500: Internal server error\n\n **Note:** The highest progress strategy is ideal when you want to preserve the most reading progress across devices. Use this when you've been\
\ reading on multiple devices and want to keep the furthest position."
- `not-started`: Books you haven't read yet (progress = 0 or no record)
## Example Request
```json
{
"collection_name": "continue-reading"
}
```
## Response
Returns success message on restore.
code:"test(\"status must be 200 with valid collection\", function() {\n expect(res.status).to.eql(200);\n});\n\ntest(\"response has success message\", function() {\n expect(res.body.message).to.exist;\n});"
docs:"Restore a system collection to its default state.\n\n**Endpoint**: POST /api/dashboard/restore-system-collection\n**Auth**: Required (Bearer token)\n\n## Request Body\n\n| Field | Type | Required | Description |\n|-------|------|----------|-------------|\n| collection_name | string | Yes | Name of system collection to restore |\n\nValid collection names:\n- `continue-reading`: Books you're currently reading (0 < progress < 1)\n- `recently-added`: Newly added items to this library\n- `recently-read`: Books you've finished (progress >= 1)\n- `not-started`: Books you haven't read yet (progress = 0 or no record)\n\n## Example Request\n\n```json\n{\n \"collection_name\": \"continue-reading\"\n}\n```\n\n## Response\n\nReturns success message on restore."
code:"test(\"status must be 200 with valid request\", function() {\n expect(res.status).to.eql(200);\n});"
docs:"Update dashboard preferences for the authenticated user.\n\n**Endpoint**: PUT /api/dashboard/preferences\n**Auth**: Required (Bearer token)\n\n## Request Body\n\n| Field | Type | Required | Description |\n|-------|------|----------|-------------|\n| library_id | string | Yes | Library UUID |\n| hidden_collections | array | No | Collection names to hide from dashboard |\n| collection_order | array | No | Collection names in custom display order |\n| items_per_section | int | No | Items to show per section (default: 20) |\n\n## Example Request\n\n```json\n{\n \"library_id\": \"cc23c3a7-f8fb-451a-a78d-2a16df1b725a\",\n \"hidden_collections\": [\"not-started\"],\n \"collection_order\": [\"recently-added\", \"continue-reading\", \"recently-read\"],\n \"items_per_section\": 20\n}\n```\n\n## Response\n\nReturns updated preferences object on success."
- `limit` (number, optional): Maximum number of books to auto-link (default: 50)
**Response:**
- `results` (array): Results for each auto-link attempt
- `total` (number): Total number of books processed
- `success` (number): Number of successful links
- `failed` (number): Number of failed links
**Status Codes:**
- 200: Success
- 400: Invalid request data
- 401: Unauthorized
- 500: Internal server error
**Note:** Higher confidence thresholds produce fewer but more accurate matches. Consider the tradeoff between automation and accuracy.
docs: "## Auto-Link Unlinked Books\n\n Automatically links unlinked books to media items based on title and author matching with a configurable confidence threshold.\n\n **Method:** POST\n\n **Endpoint:** /sync/auto-link-books\n\n **Authentication:** Bearer token\n\n **Request Body:**\n - `confidence_threshold` (number, optional): Minimum confidence score for auto-linking (0-1, default: 0.8)\n - `limit` (number, optional): Maximum number of books to auto-link (default: 50)\n\n **Response:**\n - `results` (array): Results for each auto-link attempt\n - `total` (number): Total number of books processed\n - `success` (number): Number of successful links\n - `failed` (number): Number of failed links\n\n **Status Codes:**\n - 200: Success\n - 400: Invalid request data\n - 401: Unauthorized\n - 500: Internal server error\n\n **Note:** Higher confidence thresholds produce fewer but more accurate matches. Consider the tradeoff between automation and accuracy."
Links multiple unlinked books to media items in a single request.
**Method:** POST
**Endpoint:** /sync/bulk-link-books
**Authentication:** Bearer token
**Request Body:**
- `links` (array): Array of link objects
- `unlinked_book_id` (string): Unlinked book UUID
- `media_item_id` (string): Media item UUID to link to
- `confidence_score` (number): Match confidence (0-1)
**Response:**
- `results` (array): Results for each link attempt
- `total` (number): Total number of links processed
- `success` (number): Number of successful links
- `failed` (number): Number of failed links
**Status Codes:**
- 200: Success
- 400: Invalid request data
- 401: Unauthorized
- 500: Internal server error
**Note:** Use this endpoint after reviewing suggestions from the Get Unlinked Book Suggestions endpoint.
docs:"## Bulk Link Unlinked Books\n\n Links multiple unlinked books to media items in a single request.\n\n **Method:** POST\n\n **Endpoint:** /sync/bulk-link-books\n\n **Authentication:** Bearer token\n\n **Request Body:**\n - `links` (array): Array of link objects\n - `unlinked_book_id` (string): Unlinked book UUID\n - `media_item_id` (string): Media item UUID to link to\n - `confidence_score` (number): Match confidence (0-1)\n\n **Response:**\n - `results` (array): Results for each link attempt\n - `total` (number): Total number of links processed\n - `success` (number): Number of successful links\n - `failed` (number): Number of failed links\n\n **Status Codes:**\n - 200: Success\n - 400: Invalid request data\n - 401: Unauthorized\n - 500: Internal server error\n\n **Note:** Use this endpoint after reviewing suggestions from the Get Unlinked Book Suggestions endpoint."
- `Bookmarks` (array, optional): Array of bookmark objects
- `BookmarkId`: Unique bookmark ID
- `ContentId`: Book UUID
- `BookmarkText`: Bookmark text/note
- `BookmarkType`: Type (bookmark, annotation, etc.)
- `BookmarkTitle`: Bookmark title
- `Highlights` (array, optional): Array of highlight objects (same structure as bookmarks)
**Response:**
- Sync result confirmation
**Status Codes:**
- 200: Success
- 401: Unauthorized
- 500: Internal server error
**Note:** Allows Bookhoard server to push updates to Kobo device, including reading progress, bookmarks, and highlights.
docs:"## Sync from Bookhoard to Kobo\n\n Server-initiated sync pushing progress, bookmarks, and highlights from Bookhoard to Kobo device. Two-way sync endpoint.\n\n **Method:** POST\n\n **Endpoint:** /api/sync/kobo/sync-from-server\n\n **Authentication:** Bearer token with Kobo device identification\n\n **Headers:**\n - `x-kobo-device` (string): JSON string containing Kobo device info\n - `DeviceId`: Kobo device ID\n - `Model`: Kobo device model\n - `SerialNumber`: Kobo device serial number\n\n **Request Body:** Array of sync data objects\n - `ContentId` (string): Book UUID\n - `PercentRead` (number): Reading progress percentage (0-100)\n - `LastModified` (string): ISO 8601 timestamp\n - `Bookmarks` (array, optional): Array of bookmark objects\n - `BookmarkId`: Unique bookmark ID\n - `ContentId`: Book UUID\n - `BookmarkText`: Bookmark text/note\n - `BookmarkType`: Type (bookmark, annotation, etc.)\n - `BookmarkTitle`: Bookmark title\n - `Highlights` (array,\
\ optional): Array of highlight objects (same structure as bookmarks)\n\n **Response:**\n - Sync result confirmation\n\n **Status Codes:**\n - 200: Success\n - 401: Unauthorized\n - 500: Internal server error\n\n **Note:** Allows Bookhoard server to push updates to Kobo device, including reading progress, bookmarks, and highlights."
docs:"## Sync Books from Server to Kobo\n\nPulls reading progress, annotations, and bookmarks from server to Kobo device.\n\n**Method:** POST\n\n**Endpoint:** /api/sync/kobo/sync-from-server\n\n**Authentication:** Bearer token\n\n**Request Body:**\n- `force_sync` (boolean): Force full sync (ignore last sync time)\n- `books` (array, optional): List of book UUIDs to sync\n - If empty, syncs all books with server data\n - If specified, only syncs listed books\n- `sync_options` (object, optional):\n - `since_date` (string): ISO 8601 timestamp for incremental sync\n - `include_annotations` (boolean): Include highlights/notes\n - `include_progress` (boolean): Include reading progress\n - `conflict_resolution` (string): `most_recent`, `server_wins`, `client_wins`\n\n**Response:**\n- `synced` (integer): Number of books synced\n- `skipped` (integer): Books with no server changes\n- `failed` (integer): Books that failed to sync\n- `results` (array): Per-book sync results\n - `book_id` (string):\
\ Book UUID\n - `status` (string): `synced`, `skipped`, `failed`\n - `progress_pulled` (boolean): Whether progress was downloaded\n - `annotations_pulled` (integer): Number of annotations downloaded\n - `error` (string, optional): Error message if failed\n- `server_timestamp` (string): Server timestamp of sync\n\n**Status Codes:**\n- 200: Sync completed\n- 207: Multi-status (some succeeded, some failed)\n- 401: Unauthorized\n- 400: Invalid request\n\n**Sync Direction:**\n- **Server → Device** (this endpoint)\n- Device → Server: Use `/sync/kobo/markup` endpoint\n- Bidirectional sync achieved by calling both\n\n**Pull Sync Use Cases:**\n- New device setup (download all progress)\n- Device replacement (restore from server)\n- Multi-device sync (pull changes from other devices)\n- Conflict resolution (server wins)\n- Manual \"Download from server\" operation\n\n**Conflict Resolution:**\n- `most_recent`: Latest timestamp wins (default)\n- `server_wins`: Server data always used\n- `client_wins`:\
\ Device data preserved\n- Applied per-book, per-item\n\n**Sync Optimization:**\n- Incremental sync by default (since last sync)\n- Force sync does full comparison\n- Book-level batching (10 books per batch)\n- Delta transfer (only changed items)\n- Compression for large annotation sets\n\n**Kobo Device Behavior:**\n- Device updates local database\n- Progress reflected in reading view\n- Annotations appear in Notebook\n- Bookmarks updated in navigation\n- Conflict warnings shown to user\n- Sync progress displayed on screen\n\n**Performance:**\n- Small sync (1-10 books): 2-5 seconds\n- Medium sync (10-50 books): 5-15 seconds\n- Large sync (50-200 books): 15-45 seconds\n- Timeout: 120 seconds\n\n**Use Cases:**\n- Initial device sync\n- After firmware update\n- From another device's changes\n- Manual sync request\n- Conflict recovery\n- Data restoration"
docs:"## Sync Multiple Books Progress\n\nSynchronizes reading progress for multiple books in a single request.\n\n**Method:** POST\n\n**Endpoint:** /api/sync/kobo/markup\n\n**Authentication:** Bearer token\n\n**Headers:**\n- `Authorization`: Bearer {{kobo_device_token}}\n- `x-kobo-device`: Device information (Model: \"Kobo Aura\")\n\n**Request Body:**\n- `ReadingSync` (array): Multiple progress items\n - Each item contains: ContentId, PercentRead, EntitlementId, etc.\n- `BookmarkSync` (array): Empty for progress-only batch\n\n**Response:**\n- `total` (integer): Total items in request\n- `synced` (integer): Successfully synced\n- `failed` (integer): Failed items\n- `results` (array): Per-item results\n - `ContentId` (string): Book UUID\n - `status` (string): `synced`, `failed`, `skipped`\n - `error` (string, optional): Error message if failed\n\n**Status Codes:**\n- 200: Batch sync completed\n- 207: Multi-status (some succeeded, some failed)\n- 401: Unauthorized\n- 413: Payload too large\
\ (>1MB)\n\n**Batch Sync Advantages:**\n- Efficient sync of entire library\n- Reduces HTTP overhead\n- Faster for devices with many books\n- Atomic operation (all or nothing by default)\n\n**Kobo Batch Sync Behavior:**\n- Triggered when device connects after being offline\n- Occurs during manual \"Sync now\" operation\n- Limited to 100 books per request\n- Progress updates shown on device\n- Failed items retried individually\n\n**Performance:**\n- Typical batch: 10-50 books in 1-3 seconds\n- Large batch: 50-100 books in 3-8 seconds\n- Timeout: 30 seconds\n- Rate limit: 10 batches per minute per device\n\n**Use Cases:**\n- Initial device sync after registration\n- Catch-up sync after extended offline period\n- Library-wide progress update\n- Pre-sync before device firmware update"
docs:"## Sync Progress with Bookmarks\n\nSynchronizes reading progress and highlights/annotations from Kobo device.\n\n**Method:** POST\n\n**Endpoint:** /api/sync/kobo/markup\n\n**Authentication:** Bearer token\n\n**Headers:**\n- `Authorization`: Bearer {{kobo_device_token}}\n- `x-kobo-device`: Device information JSON\n\n**Request Body:**\n- `ReadingSync` (array): Progress items (see Sync Reading Progress)\n- `BookmarkSync` (array): Highlights and annotations\n - `ContentId` (string): Book UUID\n - `BookmarkText` (string): Highlighted text or note content\n - `BookmarkType` (string): Type of bookmark\n - `annotation`: Highlighted text\n - `note`: Personal note\n - `bookmark`: Location bookmark\n - `BookmarkTitle` (string): Reference (e.g., chapter name)\n - `ChapterID` (string, optional): Chapter identifier\n - `DateCreated` (string, optional): Creation timestamp\n\n**Response:**\n- `progress_synced` (integer): Progress items synced\n- `bookmarks_synced` (integer): Bookmark\
\ items synced\n- `conflicts_resolved` (integer): Number of conflicts auto-resolved\n- `timestamp` (string): Sync timestamp\n- `details` (object): Sync breakdown by type\n\n**Status Codes:**\n- 200: Successful sync\n- 401: Unauthorized\n- 400: Invalid data\n\n**Kobo Highlight Features:**\n- 5 highlight colors (yellow, green, blue, pink, orange)\n- Chapter-based organization\n- Linked to reading progress\n- Appears in Kobo \"Notebook\" view\n- Can be exported from device\n- Syncs across all user devices\n\n**Annotation Sync:**\n- Highlight text preserved exactly\n- Color mapped to system colors\n- Chapter reference maintained\n- Location data converted to standard format\n- Notes attached to highlights synced separately"
docs:"## Sync Single Bookmark\n\nSynchronizes an individual bookmark/highlight from Kobo device.\n\n**Method:** POST\n\n**Endpoint:** /api/sync/kobo/bookmark\n\n**Authentication:** Bearer token\n\n**Headers:**\n- `Authorization`: Bearer {{kobo_device_token}}\n- `x-kobo-device`: Device information JSON\n\n**Request Body:**\n- `ContentId` (string): Book UUID\n- `BookmarkText` (string): Highlighted text or bookmark description\n- `BookmarkType` (string): Type of bookmark\n - `annotation`: Highlighted text passage\n - `note`: Personal note\n - `bookmark`: Location marker\n- `BookmarkTitle` (string): Reference title (e.g., chapter name)\n- `ChapterID` (string, optional): Chapter identifier\n- `DateCreated` (string, optional): ISO 8601 timestamp\n- `highlight_color` (string, optional): Color name (yellow, green, blue, pink, orange)\n\n**Response:**\n- `id` (string): Server bookmark ID\n- `ContentId` (string): Associated book UUID\n- `status` (string): `created`, `updated`, `skipped` (duplicate)\n\
- `timestamp` (string): Server timestamp\n- `url` (string): API URL to retrieve bookmark\n\n**Status Codes:**\n- 201: Bookmark created\n- 200: Bookmark updated (duplicate found)\n- 409: Duplicate bookmark (unchanged)\n- 401: Unauthorized\n- 400: Invalid bookmark data\n\n**Single Bookmark Sync vs Batch:**\n- **Single bookmark endpoint:** Real-time, immediate sync\n- **Batch markup endpoint:** Deferred, periodic sync\n- Use single when user explicitly creates highlight\n- Use batch for periodic background sync\n\n**Kobo Trigger:**\n- User highlights text → immediate single sync\n- User adds note → immediate single sync\n- Device goes online → batch sync of all changes\n\n**Duplicate Detection:**\n- Same ContentId + similar BookmarkText + same location\n- Updates existing if text modified\n- Skips if identical bookmark exists\n- Preserves creation date of original\n\n**Use Cases:**\n- Real-time highlight sync\n- Instant note backup\n- Immediate annotation sharing\n- Quick single annotation\
\ update\n- Testing annotation sync\n\n**Notes:**\n- Much faster than full markup sync\n- Lower bandwidth usage\n- Ideal for intermittent connectivity\n- Can be called multiple times safely"
docs:"## Sync with Multiple Annotations\n\nSynchronizes reading progress with multiple highlights and notes.\n\n**Method:** POST\n\n**Endpoint:** /api/sync/kobo/markup\n\n**Authentication:** Bearer token\n\n**Headers:**\n- `Authorization`: Bearer {{kobo_device_token}}\n- `x-kobo-device`: Device info (Model: \"Kobo Libra\")\n\n**Request Body:**\n- `ReadingSync` (array): Single book progress\n- `BookmarkSync` (array): Multiple annotations\n - Can include highlights (annotation type)\n - Can include notes (note type)\n - Each has: ContentId, BookmarkText, BookmarkType, BookmarkTitle\n\n**Response:**\n- `progress_synced` (boolean): Progress update status\n- `annotations_synced` (integer): Number of annotations synced\n- `highlights_count` (integer): Highlights synced\n- `notes_count` (integer): Notes synced\n- `conflicts` (array): Any annotation conflicts resolved\n- `timestamp` (string): Sync completion time\n\n**Status Codes:**\n- 200: Successful sync\n- 401: Unauthorized\n- 400: Invalid\
\ annotation data\n\n**Kobo Annotation Types:**\n- **Highlights:** Selected text passages\n - 5 preset colors available\n - Can have chapter titles\n - Exportable to PDF/Mobile\n- **Notes:** Personal annotations\n - Free-form text\n - Can be attached to highlights\n - Separate from highlights\n- **Bookmarks:** Location markers\n - Chapter positions\n - Quick navigation\n\n**Sync Behavior:**\n- Duplicates detected by content matching\n- Most recent edit wins conflicts\n- Annotations linked to book content\n- Chapter references preserved\n- Order maintained from device\n\n**Kobo Notebook Export:**\n- All annotations appear in Kobo \"Notebook\"\n- Can be exported to PDF\n- Can be exported to Mobile (text)\n- Organized by book\n- Shows highlight context\n\n**Use Cases:**\n- Study and research\n- Book club discussion prep\n- Content review\n- Sharing insights\n- Personal learning archive"
Manually links an unlinked book to a media item in the library.
**Method:** POST
**Endpoint:** /api/sync/link-book
**Authentication:** Bearer token
**Request Body:**
- `unlinked_book_id` (string): Unlinked book UUID
- `media_item_id` (string): Media item UUID to link to
- `confidence_score` (number): Match confidence (0-1, 1.0 for manual)
**Response:**
- `status` (string): Link status (linked)
- `unlinked_book_id` (string): Unlinked book UUID
- `media_item_id` (string): Media item UUID
- `message` (string): Success message
**Status Codes:**
- 200: Success - book linked
- 400: Invalid request
- 401: Unauthorized
- 404: Book or media item not found
- 500: Internal server error
docs:"## Link Unlinked Book - Manual Resolution\n\n Manually links an unlinked book to a media item in the library.\n\n **Method:** POST\n\n **Endpoint:** /api/sync/link-book\n\n **Authentication:** Bearer token\n\n **Request Body:**\n - `unlinked_book_id` (string): Unlinked book UUID\n - `media_item_id` (string): Media item UUID to link to\n - `confidence_score` (number): Match confidence (0-1, 1.0 for manual)\n\n **Response:**\n - `status` (string): Link status (linked)\n - `unlinked_book_id` (string): Unlinked book UUID\n - `media_item_id` (string): Media item UUID\n - `message` (string): Success message\n\n **Status Codes:**\n - 200: Success - book linked\n - 400: Invalid request\n - 401: Unauthorized\n - 404: Book or media item not found\n - 500: Internal server error"
docs:"## Get KOReader Book Metadata\n\nRetrieves metadata for a specific book from the server.\n\n**Method:** GET\n\n**Endpoint:** /api/sync/koreader/metadata/:book_uuid\n\n**Authentication:** Bearer token\n\n**Path Parameters:**\n- `book_uuid` (string): SHA-256 based book identifier\n\n**Response:**\n- `uuid` (string): Book UUID (SHA-256)\n- `title` (string): Book title\n- `authors` (array): Author names\n- `series` (string, optional): Series name\n- `series_index` (number, optional): Position in series\n- `publisher` (string, optional): Publisher name\n- `publication_date` (string, optional): Release date\n- `language` (string, optional): ISO 639-1 language code\n- `description` (string, optional): Book description\n- `cover_url` (string, optional): Cover image URL\n- `cover_thumbnail_url` (string, optional): Thumbnail URL\n- `identifiers` (object): Various identifiers\n - `isbn` (string, optional): ISBN-13\n - `asin` (string, optional): Amazon ASIN\n - `goodreads` (string, optional):\
\ Goodreads ID\n - `google_books` (string, optional): Google Books ID\n- `metadata_sources` (array): Sources metadata was pulled from\n- `last_synced` (string): Last metadata sync timestamp\n- `file_metadata` (object):\n - `file_size` (integer): File size in bytes\n - `format` (string): File format (epub, pdf, mobi, etc.)\n - `pages` (integer, optional): Page count\n - `word_count` (integer, optional): Estimated words\n\n**Status Codes:**\n- 200: Success\n- 401: Unauthorized\n- 404: Book not found\n\n**Metadata Purpose:**\n- Enrich book information on device\n- Improve book organization\n- Enable better search\n- Support series sorting\n- Provide cover images\n- Link to external sources\n\n**KOReader Usage:**\n- Display in book info dialog\n- Used for library sorting\n- Shown in file browser\n- Series grouping\n- Cover display\n- Search optimization\n\n**SHA-256 Book ID:**\n- Primary identifier in KOReader\n- Universal across devices\n- Format-independent\n- Generated from file content\n\
- Survives metadata changes\n\n**Metadata Sources:**\n- Google Books API\n- Open Library\n- Goodreads API\n- ISBN database lookup\n- Publisher metadata\n- User-provided metadata\n\n**Use Cases:**\n- Initial book import\n- Metadata refresh\n- Cover image download\n- Series organization\n- Duplicate detection\n- Library management"
docs:"## Checkpoint Sync\n\nIncremental sync using checkpoint-based change tracking.\n\n**Method:** POST\n\n**Endpoint:** /api/sync/koreader/progress\n\n**Authentication:** Bearer token\n\n**Request Body:**\n- `sync_mode` (string): Must be `checkpoint`\n- `checkpoint_id` (string): Unique checkpoint identifier\n- `since_timestamp` (string): ISO 8601 timestamp for incremental sync\n- `books` (array): Book progress items\n - `uuid` (string): Book UUID\n - `percentage` (number): Progress 0.0-1.0\n - `chapter` (integer): Current chapter\n - `epubcfi` (string, optional): Current position\n - `modified_since` (boolean, optional): Whether modified since checkpoint\n\n**Response:**\n- `checkpoint_id` (string): Server checkpoint ID\n- `checkpoint_timestamp` (string): Checkpoint creation time\n- `processed` (integer): Books processed\n- `changes_only` (boolean): Whether only changed items synced\n- `next_checkpoint_id` (string): ID for next checkpoint sync\n- `results` (array): Per-book results\n\
\n**Status Codes:**\n- 200: Checkpoint sync completed\n- 401: Unauthorized\n- 400: Invalid checkpoint or timestamp\n\n**Checkpoint Sync Benefits:**\n- **Incremental**: Only sync changed items\n- **Efficient**: Smaller payloads\n- **Fast**: Reduced processing time\n- **Reliable**: Checkpoint-based tracking\n- **Resumable**: Can continue from last checkpoint\n\n**Checkpoint Mechanism:**\n- Server tracks changes since checkpoint\n- Client provides checkpoint ID or timestamp\n- Only modified books returned/processed\n- Checkpoint ID advances on each sync\n- Supports large libraries efficiently\n\n**Use Cases:**\n- Large libraries (100+ books)\n- Intermittent connectivity\n- Bandwidth optimization\n- Battery conservation\n- Background sync\n- Periodic sync (every 5-15 minutes)\n\n**Checkpoint Lifecycle:**\n1. Initial sync: No checkpoint (full sync)\n2. Server returns checkpoint_id\n3. Next sync: Client sends checkpoint_id\n4. Server processes only changes\n5. New checkpoint_id returned\n6.\
\ Repeat from step 3\n\n**Sync Optimization:**\n- Only books with progress changes\n- Skips unmodified books\n- Delta transfer\n- Compression for large payloads\n- Batch processing\n\n**Failure Handling:**\n- Checkpoint ID preserved on failure\n- Retry with same checkpoint\n- Full sync if checkpoint expired\n- Checkpoint validity: 24 hours\n- Auto-fallback to full sync\n\n**Performance:**\n- Small changes (1-10 books): 100-300ms\n- Medium changes (10-50 books): 300ms-1s\n- Large changes (50-100 books): 1-3s\n- Typical: 5-10x faster than full sync\n\n**When to Use:**\n- Default sync mode for most users\n- Periodic background sync\n- Large library management\n- Mobile/network-constrained environments\n- Battery-powered devices\n\n**Configuration:**\n- Checkpoint expiration: 24 hours\n- Max history: 100 checkpoints\n- Auto-cleanup of old checkpoints\n- Configurable sync interval"
http:
method:POST
url:'{{base_url}}/api/sync/koreader/progress'
auth:inherit
headers:
- key:Content-Type
value:application/json
body:
type:json
json:
sync_mode:checkpoint
checkpoint_id:checkpoint-uuid
since_timestamp:'2026-01-30T19:00:00Z'
books:
- uuid:book-uuid-1
percentage:0.45
chapter:3
- uuid:book-uuid-2
percentage:0.75
chapter:8
docs:|-
## Checkpoint Sync
Incremental sync using checkpoint-based change tracking.
docs:"## Immediate Sync - Page Turn\n\nReal-time progress sync triggered immediately on page turn.\n\n**Method:** POST\n\n**Endpoint:** /api/sync/koreader/progress\n\n**Authentication:** Bearer token\n\n**Request Body:**\n- `sync_mode` (string): Must be `immediate`\n- `books` (array): Current book progress\n - `uuid` (string): Book UUID\n - `percentage` (number): Precise progress (0.45678 = 45.678%)\n - `chapter` (integer): Current chapter\n - `timestamp` (string): ISO 8601 timestamp\n - `epubcfi` (string, optional): Current position\n - `page` (integer, optional): Current page\n\n**Response:**\n- `synced` (boolean): Immediate sync status\n- `timestamp` (string): Server timestamp\n- `next_sync_suggested` (string): Suggested next sync time\n\n**Status Codes:**\n- 200: Sync queued\n- 202: Accepted for processing\n- 401: Unauthorized\n- 429: Too many immediate sync requests (rate limited)\n\n**Immediate Sync Mode:**\n- **Purpose**: Real-time progress updates\n- **Trigger**: Every page\
\ turn (configurable)\n- **Priority**: High priority processing\n- **Latency**: <100ms typical\n- **Best effort**: May be queued under load\n\n**Rate Limiting:**\n- Max 60 requests per minute per device\n- Throttled after limit reached\n- Suggests switching to periodic sync\n- Prevents server overload\n\n**Use Cases:**\n- Real-time multi-device reading\n- Live progress sharing\n- Instant position backup\n- Critical reading points\n- Test/profiling mode\n\n**Performance:**\n- Ultra-fast sync\n- Minimal payload\n- Optimized for speed\n- Async processing\n- No confirmation wait\n\n**Battery Considerations:**\n- More frequent network use\n- Higher battery consumption\n- WiFi recommended\n- Can reduce sync frequency in settings\n\n**Configuration:**\n- Can enable/disable per device\n- Adjustable frequency (every page, every N pages)\n- Automatic fallback to periodic sync on error\n- Respects device power-save mode\n\n**When to Use:**\n- Critical reading sessions\n- Multi-device concurrent reading\n\
- Research and study\n- Testing sync functionality\n- When power source available\n\n**When NOT to Use:**\n- Battery conservation needed\n- Unstable network\n- Extended reading sessions\n- Background sync preferred"
http:
method:POST
url:'{{base_url}}/api/sync/koreader/progress'
auth:inherit
headers:
- key:Content-Type
value:application/json
body:
type:json
json:
sync_mode:immediate
books:
- uuid:book-uuid
percentage:0.45678
chapter:3
timestamp:'2026-01-30T20:00:00Z'
docs:|-
## Immediate Sync - Page Turn
Real-time progress sync triggered immediately on page turn.
- 401: Unauthorized\n- 400: Invalid data\n\n**Dedicated Bookmark Endpoint:**\n- **Purpose**: Sync bookmarks independently\n- **Use case**: More frequent bookmark updates\n- **Advantage**: Separate from progress sync\n- **Efficiency**: Smaller payloads\n\n**KOReader Bookmark Features:**\n- Chapter-based organization\n- Quick navigation markers\n- Hierarchical bookmarks (via plugins)\n- Custom bookmark titles\n- Date/time tracking\n- EPUB CFI precision\n\n**Bookmark Types in KOReader:**\n- **Location bookmarks**: Quick navigation points\n- **Chapter marks**: Auto-generated chapter markers\n- **Progress bookmarks**: Last read positions\n- **Custom bookmarks**: User-created markers\n- **Search bookmarks**: Saved search results\n\n**Sync Behavior:**\n- Duplicate detection by position + text\n- Most recent wins on conflicts\n- Chapter order preserved\n- Auto-generated vs manual bookmarks differentiated\n- Merge with existing bookmarks\n\n**KOReader Device Integration:**\n- Created via \"Add\
\ bookmark\" menu\n- Shown in \"Bookmarks\" panel\n- Quick access via \"Go to bookmark\"\n- Exportable to JSON\n- Can be edited/deleted\n\n**Use Cases:**\n- Quick navigation aids\n- Chapter markers\n- Important passages\n- Reading progress points\n- Study session markers"
http:
method:POST
url:'{{base_url}}/api/sync/koreader/bookmarks'
auth:inherit
headers:
- key:Content-Type
value:application/json
body:
type:json
json:
library_id:optional-library-uuid
books:
- uuid:book-uuid
bookmarks:
- chapter:3
datetime:'2026-01-30T19:55:00Z'
notes:Marked this chapter as important
pos0:'epubcfi(/6/4/2:15)'
pos1:'epubcfi(/6/4/2:20)'
page:45
text:Important passage
type:bookmark
docs:|-
## Sync KOReader Bookmarks
Synchronizes bookmarks separately from progress for KOReader.
**Method:** POST
**Endpoint:** /api/sync/koreader/bookmarks
**Authentication:** Bearer token
**Request Body:**
- `library_id` (string, optional): Library UUID
- `books` (array): Books with bookmarks
- `uuid` (string): Book UUID
- `bookmarks` (array): Bookmark items
- `chapter` (integer): Chapter number
- `datetime` (string): ISO 8601 timestamp
- `notes` (string): Bookmark description
- `pos0` (string): EPUB CFI start position
- `pos1` (string): EPUB CFI end position
- `page` (integer): Page number
- `text` (string): Displayed text
- `type` (string): `bookmark`, `highlight`, or `note`
docs:"## Sync KOReader Highlights\n\nSynchronizes text highlights separately from other annotations.\n\n**Method:** POST\n\n**Endpoint:** /api/sync/koreader/highlights\n\n**Authentication:** Bearer token\n\n**Request Body:**\n- `library_id` (string, optional): Library UUID\n- `books` (array): Books with highlights\n - `uuid` (string): Book UUID\n - `highlights` (array): Highlight items\n - `datetime` (string): Creation timestamp\n - `text` (string): Highlighted text content\n - `chapter` (integer): Chapter number\n - `pos0` (string): EPUB CFI start position\n - `pos1` (string): EPUB CFI end position\n - `page_start` (integer): Start page\n - `page_end` (integer): End page\n - `color` (string, optional): Color name or hex\n - `note` (string, optional): Attached note\n\n**Response:**\n- `synced` (integer): Highlights synced\n- `duplicates_skipped` (integer): Duplicates found\n- `with_notes` (integer): Highlights that have notes attached\n- `timestamp` (string):\
\ Sync timestamp\n\n**KOReader Highlight Features:**\n- Precise text selection (EPUB CFI)\n- Custom color support (via plugins)\n- Multi-page highlights\n- Chapter references\n- Timestamps for sorting\n- Attached notes support\n- Full text preservation\n\n**Highlight Colors (via plugins):**\n- Yellow (default): General highlighting\n- Green: Important concepts\n- Blue: Key information\n- Red/pink: Critical content\n- Orange: Interesting quotes\n- Custom RGB colors available\n\n**EPUB CFI Advantages:**\n- Precise start/end positions\n- Works across font size changes\n- Survives text reflow\n- Device-independent\n- Standardized format\n\n**Dedicated Highlight Endpoint:**\n- **Separate from progress**: Sync highlights independently\n- **Smaller payload**: Just highlights, no progress\n- **More frequent**: Can sync highlights immediately\n- **Focused**: Single-purpose endpoint\n\n**Sync Behavior:**\n- Exact text matching for duplicates\n- Position-based conflict resolution\n- Color preservation\
\ across devices\n- Note attachments preserved\n- Order maintained by position\n\n**KOReader Device Features:**\n- Created via long-press or selection\n- Color picker available\n- Can add notes immediately\n- Shows in \"Highlights\" panel\n- Exportable to Evernote, etc.\n- Searchable by content\n\n**Use Cases:**\n- Study and research\n- Content curation\n- Quote collection\n- Academic work\n- Sharing insights"
http:
method:POST
url:'{{base_url}}/api/sync/koreader/highlights'
auth:inherit
headers:
- key:Content-Type
value:application/json
body:
type:json
json:
library_id:optional-library-uuid
books:
- uuid:book-uuid
highlights:
- datetime:'2026-01-30T19:50:00Z'
text:Important quote from book
chapter:4
pos0:'epubcfi(/6/4/2:20)'
pos1:'epubcfi(/6/4/2:30)'
page_start:78
page_end:79
docs:|-
## Sync KOReader Highlights
Synchronizes text highlights separately from other annotations.
**Method:** POST
**Endpoint:** /api/sync/koreader/highlights
**Authentication:** Bearer token
**Request Body:**
- `library_id` (string, optional): Library UUID
- `books` (array): Books with highlights
- `uuid` (string): Book UUID
- `highlights` (array): Highlight items
- `datetime` (string): Creation timestamp
- `text` (string): Highlighted text content
- `chapter` (integer): Chapter number
- `pos0` (string): EPUB CFI start position
- `pos1` (string): EPUB CFI end position
- `page_start` (integer): Start page
- `page_end` (integer): End page
- `color` (string, optional): Color name or hex
- `note` (string, optional): Attached note
**Response:**
- `synced` (integer): Highlights synced
- `duplicates_skipped` (integer): Duplicates found
- `with_notes` (integer): Highlights that have notes attached
- `timestamp` (string): Sync timestamp
**KOReader Highlight Features:**
- Precise text selection (EPUB CFI)
- Custom color support (via plugins)
- Multi-page highlights
- Chapter references
- Timestamps for sorting
- Attached notes support
- Full text preservation
**Highlight Colors (via plugins):**
- Yellow (default): General highlighting
- Green: Important concepts
- Blue: Key information
- Red/pink: Critical content
- Orange: Interesting quotes
- Custom RGB colors available
**EPUB CFI Advantages:**
- Precise start/end positions
- Works across font size changes
- Survives text reflow
- Device-independent
- Standardized format
**Dedicated Highlight Endpoint:**
- **Separate from progress**: Sync highlights independently
- **Smaller payload**: Just highlights, no progress
- **More frequent**: Can sync highlights immediately
docs:"## Sync KOReader Notes\n\nSynchronizes user notes separately from highlights and bookmarks.\n\n**Method:** POST\n\n**Endpoint:** /api/sync/koreader/notes\n\n**Authentication:** Bearer token\n\n**Request Body:**\n- `library_id` (string, optional): Library UUID\n- `books` (array): Books with notes\n - `uuid` (string): Book UUID\n - `notes` (array): Note items\n - `datetime` (string): Creation/modification timestamp\n - `text` (string): Note content\n - `chapter` (integer): Chapter number\n - `pos0` (string, optional): Related EPUB CFI position\n - `page` (integer, optional): Page number\n - `highlighted_text` (string, optional): Related highlight\n\n**Response:**\n- `synced` (integer): Notes synced\n- `duplicates_skipped` (integer): Duplicate notes found\n- `attached_to_highlights` (integer): Notes linked to highlights\n- `timestamp` (string): Sync timestamp\n\n**KOReader Note Features:**\n- Free-form text notes\n- Can be standalone or attached to highlights\n- Chapter-based\
\ organization\n- Timestamped for sorting\n- Markdown support (some versions)\n- Longer form content\n- No character limit\n\n**Note Types:**\n- **Standalone notes**: Independent notes about chapter/section\n- **Attached notes**: Notes attached to specific highlights\n- **Chapter notes**: Notes about entire chapter\n- **Book notes**: General notes about the book\n\n**Dedicated Notes Endpoint:**\n- **Separate sync**: Notes sync independently\n- **Flexible**: Not tied to highlights\n- **Efficient**: Smaller, focused payload\n- **Immediate**: Can sync right after note creation\n\n**Attached Notes:**\n- Linked to specific highlight\n- Shares highlight's position\n- Shown together with highlight\n- Deleted when highlight deleted (optional)\n- Color matches highlight\n\n**Sync Behavior:**\n- Text-based duplicate detection\n- Time-based conflict resolution\n- Chapter organization preserved\n- Markdown formatting preserved\n- Attachment links maintained\n\n**KOReader Device Integration:**\n- Created\
\ via \"Add note\" option\n- Edited in note editor\n- Shown in \"Notes\" panel\n- Can be organized by chapter\n- Export functionality available\n- Search support\n\n**Advanced Features:**\n- **Markdown**: Bold, italic, lists (some versions)\n- **Tags**: User-defined tags (via plugins)\n- **Links**: Internal/external links\n- **Formatting**: Rich text in newer versions\n\n**Use Cases:**\n- Study notes\n- Personal reflections\n- Academic annotations\n- Research insights\n- Book club discussion prep\n- Knowledge management"
http:
method:POST
url:'{{base_url}}/api/sync/koreader/notes'
auth:inherit
headers:
- key:Content-Type
value:application/json
body:
type:json
json:
library_id:optional-library-uuid
books:
- uuid:book-uuid
notes:
- datetime:'2026-01-30T19:52:00Z'
text:My personal note about this chapter
chapter:4
docs:|-
## Sync KOReader Notes
Synchronizes user notes separately from highlights and bookmarks.
docs:"## KOReader Sync Progress - SHA-256 Only\n\n Synchronizes reading progress using only SHA-256 hash for book identification (when UUID is not available).\n\n **Method:** POST\n\n **Endpoint:** /api/v1/koreader/sync/progress\n\n **Authentication:** Bearer token (KOReader device token)\n\n **Request Body:**\n - `sync_mode` (string): Sync mode (immediate, deferred)\n - `books` (array): Array of book progress objects\n - `sha256` (string): SHA-256 hash of book file\n - `file_path` (string): Path to book file\n - `percentage` (number): Progress percentage\n - `page` (number): Current page\n - `total_pages` (number): Total pages\n\n **Response:**\n - `sync_status` (string): Sync status\n - `books_synced` (number): Number of books synced\n\n **Status Codes:**\n - 200: Success\n - 202: Accepted\n - 401: Unauthorized\n - 500: Internal server error\n\n **Note:** Use this when book UUID is not available, falling back to SHA-256 hash for identification."
docs:"## Sync KOReader Progress - Single Book\n\nSynchronizes reading progress for a single book from KOReader device.\n\n**Method:** POST\n\n**Endpoint:** /api/sync/koreader/progress\n\n**Authentication:** Bearer token (koreader_device_token)\n\n**Request Body:**\n- `library_id` (string, optional): Library UUID for multi-library setups\n- `books` (array): Array with single book progress\n - `uuid` (string): Unique book identifier (often SHA-256 hash)\n - `title` (string): Book title\n - `authors` (array): List of authors\n - `progress` (number): Progress decimal (0.0 to 1.0)\n - `percentage` (number): Progress percentage (0.45 = 45%)\n - `last_read` (string): ISO 8601 timestamp of last read\n - `chapter` (integer): Current chapter number\n - `epubcfi` (string): EPUB Canonical Fragment Identifier\n - `character` (integer): Character position in book\n - `page` (integer, optional): Current page number\n\n**Response:**\n- `synced` (integer): Number of books synced\n- `timestamp`\
\ (string): Server sync timestamp\n- `books` (array): Sync results\n - `uuid` (string): Book UUID\n - `status` (string): `synced`, `updated`, `skipped`\n - `server_progress` (object): Server-side progress data\n\n**Status Codes:**\n- 200: Success\n- 401: Unauthorized\n- 400: Invalid request format\n\n**KOReader Progress Tracking:**\n- SHA-256 based book identification (universal across devices)\n- EPUB CFI for precise location (standard format)\n- Chapter-based navigation\n- Character-level precision\n- Supports EPUB, FB2, PDF, DJVU, MOBI formats\n\n**EPUB CFI Format:**\n- Standardized location format for EPUBs\n- Example: `epubcfi(/6/4/2:15)`\n- Identifies exact position even after reflow\n- Works across different devices/apps\n- Preserved after file modifications\n\n**Book Identification:**\n- Primary: SHA-256 hash of book file\n- Universal: Same book = same UUID across devices\n- Format-agnostic: Works for any supported format\n- Case-sensitive: Hash must match exactly\n\n**Use Cases:**\n\
\ (string): Sync timestamp\n\n**KOReader Bookmark Types:**\n- **highlight**: Selected text passages\n- **bookmark**: Location markers\n- **note**: Text annotations (can be attached to highlights)\n\n**KOReader Highlight Features:**\n- Custom colors (via color extensions)\n- Multi-color support\n- Precise EPUB CFI positioning\n- Chapter-based organization\n- Text excerpts preserved\n- Date/time stamped\n- Page number tracking\n\n**EPUB CFI in Bookmarks:**\n- `pos0`: Start position (highlight start)\n- `pos1`: End position (highlight end)\n- Exact text selection boundaries\n- Survives text reflow\n- Works across different font sizes\n\n**Sync Behavior:**\n- Duplicate detection by text + position\n- Most recent edit wins\n- Chapter references maintained\n- Order preserved from device\n- Merges with existing server bookmarks\n\n**KOReader Device Integration:**\n- Created in KOReader highlight interface\n- Shows in \"Bookmarks\" menu\n- Can be edited/deleted on device\n- Exportable to JSON/XML\n\
- Searchable by content\n\n**Use Cases:**\n- Study and research\n- Content review\n- Passage tracking\n- Quick navigation\n- Cross-device bookmark access"
http:
method:POST
url:'{{base_url}}/api/sync/koreader/progress'
auth:inherit
headers:
- key:Content-Type
value:application/json
body:
type:json
json:
library_id:optional-library-uuid
books:
- uuid:book-uuid-here
title:Book Title
authors:
- Author Name
progress:0.45
percentage:0.45
last_read:'2026-01-30T20:00:00Z'
bookmarks:
- chapter:3
datetime:'2026-01-30T19:55:00Z'
notes:highlighted text
pos0:'epubcfi(/6/4/2:15)'
pos1:'epubcfi(/6/4/2:20)'
page:45
text:highlighted text excerpt
type:highlight
docs:|-
## Sync Progress with Bookmarks
Synchronizes reading progress with bookmarks/highlights from KOReader.
**Method:** POST
**Endpoint:** /api/sync/koreader/progress
**Authentication:** Bearer token
**Request Body:**
- `library_id` (string, optional): Library UUID
- `books` (array): Books with progress and bookmarks
- `uuid`, `title`, `authors`, `progress`, etc.
- `bookmarks` (array): Bookmark items
- `chapter` (integer): Chapter number
- `datetime` (string): ISO 8601 timestamp
- `notes` (string): Note content or highlighted text
- `pos0` (string): EPUB CFI start position
- `pos1` (string): EPUB CFI end position
- `page` (integer): Page number
- `text` (string): Displayed text excerpt
- `type` (string): `highlight`, `bookmark`, or `note`
Add one or more books to a Kobo device shelf. Manages which books should be synced to a specific Kobo device. Supports multiple shelves for organization.
**Method:** POST
**Endpoint:** /api/devices/{deviceID
docs:"## Add Books to Kobo Shelf\n\n Add one or more books to a Kobo device shelf. Manages which books should be synced to a specific Kobo device. Supports multiple shelves for organization.\n\n **Method:** POST\n\n **Endpoint:** /api/devices/{deviceID"
docs:"## Check Registration Status\n\nChecks the current status of a device registration request.\n\n**Method:** POST\n\n**Endpoint:** /api/devices/register/status\n\n**Authentication:** None\n\n**Request Body:**\n- `registration_id` (string): UUID received from registration request\n\n**Response:**\n- `registration_id` (string): The registration UUID\n- `status` (string): Current status\n - `pending`: Awaiting user approval\n - `approved`: Registration approved, device ready\n - `rejected`: Registration rejected by user\n - `expired`: Registration expired (not approved in time)\n- `device_name` (string): Name of the device\n- `device_type` (string): Type of device\n- `created_at` (string): Registration timestamp\n- `updated_at` (string): Last status update timestamp\n\n**Status Codes:**\n- 200: Status retrieved successfully\n- 404: Registration ID not found\n- 400: Invalid registration ID format\n\n**Polling Recommendations:**\n- Poll every 5-10 seconds while status is `pending`\n\
- Stop polling when status changes to `approved`, `rejected`, or `expired`\n- Use exponential backoff for mobile devices to save battery"
- `media_item_id` (string, required): Media item UUID to link to
**Response:** Created file alias object
- `id` (string): Alias UUID
- `device_id` (string): Device UUID
- `file_path` (string): Device-specific file path
- `media_item_id` (string): Associated media item UUID
- `created_at` (string): Creation timestamp
- `updated_at` (string): Last update timestamp
**Status Codes:**
- 201: Created
- 200: Success
- 400: Invalid request body
- 401: Unauthorized
- 404: Device not found
- 500: Internal server error
code:"test_create_device_file_alias_success(status, headers, body) {\n if (status !== 201 && status !== 200) {\n throw new Error(\"Expected status 201 or 200, got \" + status);"
docs:"## Create Device File Alias\n\n Creates a new file alias for a device. File aliases map device-specific file paths to media items.\n\n **Method:** POST\n\n **Endpoint:** /api/devices/:id/file-aliases\n\n **Authentication:** Required (Bearer token)\n\n **Path Parameters:**\n - `id` (string, required): Device UUID\n\n **Request Body:**\n - `file_path` (string, required): Device-specific file path\n - `media_item_id` (string, required): Media item UUID to link to\n\n **Response:** Created file alias object\n - `id` (string): Alias UUID\n - `device_id` (string): Device UUID\n - `file_path` (string): Device-specific file path\n - `media_item_id` (string): Associated media item UUID\n - `created_at` (string): Creation timestamp\n - `updated_at` (string): Last update timestamp\n\n **Status Codes:**\n - 201: Created\n - 200: Success\n - 400: Invalid request body\n - 401: Unauthorized\n - 404: Device not found\n - 500: Internal server error"
Loaded 100 of 384 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
John O'Keefe