135 Commits
Author SHA1 Message Date
john-okeefe 311049379d docs(android): add QR pairing sign-in to roadmap, update auth design
Document the authentication decision reached for the Android client:
username/password login is primary (the app needs the user-JWT API
surface that device tokens cannot reach), with the app self-approving
its own device registration post-login so it still shows up on the
Devices page with sync attribution.

Add the Netflix-style QR pairing flow to the post-v1 roadmap with its
constraints: the QR grants a full login with zero typing; a typed-code
fallback covers phones with broken cameras; KOReader keeps its existing
flow (no typed codes there); and pairing must encode the configured
BASE_URL rather than a detected LAN IP so remote instances
(https://public.domain) work identically.
2026-08-28 22:45:29 -04:00
john-okeefe f65db5ab4f docs(api): align auth/devices/libraries/media-items docs with handlers
Verified against the Echo routes and handler structs, fixing drift that
would break API clients:

- login: response field is access_token, not token (AuthResponse struct)
- register status: status is only pending|approved; expiry is HTTP 410
  (not a status value), approved responses are single-use, and pending
  registrations do not survive server restarts
- visible libraries: endpoint is GET /api/libraries/visibility and
  returns a top-level array of full library rows, not a wrapped object
- media items list: response is {"data": [...]}, library_id is optional,
  limit defaults to 50 (max 1000), no total field; document the sort
  parameter, the two response shapes, and raw-vs-resolved file paths

refresh and device-registration docs verified accurate; no changes.
2026-08-28 22:15:42 -04:00
john-okeefe 7ddcdd0756 docs(readme): link Android app doc and adopt app favicon as title icon
Add the Android client design doc to the For Developers section, point
the Supported Devices table at it, and replace the emoji title icon
with the app's book-open favicon (Tokyo Night #7aa2f7) to match the
actual product branding.
2026-08-28 20:59:31 -04:00
john-okeefe b927ed9988 docs(android): add native Android client design doc
Document the planned native Android client (bookhoard-app): product
vision, tech stack and rationale (Kotlin + Compose + Readium over
hybrid/Flutter/KMP alternatives), module architecture, offline-first
sync flow over the existing REST/WebSocket API, reader and comics/manga
UX, iOS posture, distribution and licensing, and a five-milestone
roadmap. The client is a thin, offline-first consumer of the server's
existing device registration, universal progress, annotation, and
conflict-resolution APIs — no server changes required.
2026-08-28 20:59:31 -04:00
john-okeefe e2953c4a01 chore(license): relicense project from GPL-3.0 to AGPL-3.0
Replace the GPL-3.0 license text with the full GNU Affero General
Public License v3.0 text, strengthening copyleft coverage for the
network-service use case (users interacting with Bookhoard over the
network are entitled to the corresponding source).

- LICENSE: swap GPL-3.0 text for the canonical AGPL-3.0 text (gnu.org)
- README.md: update both license references (Project Status and
  License sections) from GPL-3.0 to AGPL-3.0
- docs/user/sync-guide.md: update the footer license reference

The bundled BSD 3-Clause license in internal/sevenzip/LICENSE is a
third-party dependency license and is intentionally left unchanged.
2026-08-28 20:12:20 -04:00
john-okeefe d429534b12 docs(api): deleted-annotation history + KOReader deletion propagation
Release / build-and-push (push) Successful in 2m24s
New media-items/deleted_annotations.md for the list/restore/purge
endpoints; endpoint index updated. The KOReader protocol page documents
deleted_highlights/deleted_bookmarks on the progress push and the
deletion-propagation contract: keys learned only from server pulls,
explicit arrays only (never absence), tombstone convergence via the
metadata fetch, no resurrection from stale replays, and the web history
as the restore path.
2026-08-22 13:16:54 -04:00
john-okeefe f70579b4fc feat(ui): deleted-annotation history on the book page
Replace the Notes & Highlights 'coming soon' stub with a real modal:
active counts plus a 'Recently deleted' section listing every tombstoned
highlight, note, and bookmark (type badge, deletion time in the user's
timezone, text preview), each with Restore and Delete-permanently
actions. Restore returns the annotation to every synced device; Delete
permanently is confirmed before purging. The list is server-rendered
from MediaDetail.DeletedAnnotations — no fetch on open.

Alpine handlers in book-detail.ts call the new restore/purge endpoints
and reload on success. style.css picks up the line-clamp utilities used
by the text previews.
2026-08-22 13:16:48 -04:00
john-okeefe 1f5c5a28bd feat(sync): propagate KOReader annotation deletions + history API
KOReader push (processBookAnnotations) accepts deleted_highlights and
deleted_bookmarks arrays of dedup keys and tombstones the matching rows,
after the upserts so a key present in both lists resolves to 'deleted'
(the newer intent). Deletions remain soft: rows stay restorable from the
history and echo to other devices as tombstones on their next pull. A
stale device replay of the annotation cannot resurrect the tombstone —
device pushes carry no modification timestamp, so the save loses to the
delete. Absence from these arrays is never a delete, keeping category
toggles safe.

New annotation-history endpoints (annotation_history.go, media.go):
  GET    /api/media-items/:id/annotations/deleted
  POST   /api/media-items/:id/annotations/:annotationId/restore
  DELETE /api/media-items/:id/annotations/:annotationId
All scoped to the authenticated user and the route's book; the DELETE is
the permanent purge (annotation_type required in query or body).

MediaDetail gains DeletedAnnotations, populated by the book page route
via the shared DeletedAnnotationsForBook builder, so the server-rendered
history ships with the page instead of requiring a client round-trip.

Binding tests cover the plugin's exact wire shape and the legacy
plugin case (arrays omitted -> empty).
2026-08-22 13:16:43 -04:00
john-okeefe d4c52e9a6a feat(sync): restore/purge service methods + bookmark tombstone by dedup key
RestoreAnnotationByID and PurgeAnnotationByID dispatch on annotation kind
(highlight/note/bookmark) to the new queries, broadcasting an annotation
update on restore so connected web sessions refresh. Both report whether
a row actually changed.

TombstoneBookmarkByDedupKey mirrors the existing TombstoneHighlight for
bookmarks: devices report deletions by dedup key (they have no row IDs),
and until now only highlights had a key-based tombstone path — device
bookmark deletions had nowhere to land.

ValidAnnotationKind centralizes the kind check the HTTP handlers share.
2026-08-22 13:16:36 -04:00
john-okeefe acbb6c7981 feat(db): queries for the deleted-annotation history
ListDeletedAnnotationsForBook unions tombstoned highlights, notes, and
bookmarks for a user+book regardless of the sync TTL cutoff (the history
must show everything still restorable, not just recent deletes), with
display text, secondary text, color, and both timestamps.

Restore queries clear deleted/deleted_at (lossless — the row was soft-
deleted, never removed) and are scoped to the owning user and media item
so a restore can never touch another user's annotation.

Purge queries hard-delete an already-tombstoned row: the user-driven
counterpart of the TTL maintenance sweep, for explicit 'delete
permanently' actions from the history.

All six write queries are :execrows so callers can distinguish 'restored'
from 'nothing matched' without a follow-up read.
2026-08-22 13:16:31 -04:00
john-okeefe 91c8be8562 docs(api): document the KOReader resolve endpoint
Add koreader/resolve_book.md for GET /api/sync/koreader/resolve, list
the endpoint in the API reference, and describe the resolve-then-pull-
then-push linking flow in the KOReader protocol page — including why a
device pushing to bootstrap its identity creates progress conflicts for
books already mid-read from other sources.
2026-08-22 09:57:44 -04:00
john-okeefe 6859f81144 feat(sync): add read-only KOReader book resolve endpoint
GET /api/sync/koreader/resolve?sha256={hash} maps a file content hash to
the book's UUID through the shared format-aware BookResolver (primary
media_items hash, then per-format hashes so converted KEPUB/PDF files
match) without touching any progress state.

Devices need the UUID to pull metadata, but a freshly downloaded book has
none cached. The old way of learning it was to push once, which
transmitted the device's first-page position and manufactured a progress
conflict for books already mid-read from another source. A read-only
lookup lets clients link (and pull) without ever pushing bootstrap
progress: resolve, then pull, then push.

Returns 200 {book_uuid, sha256, title, author}, 400 for a missing or
malformed hash, 404 when no library item matches.
2026-08-22 09:57:35 -04:00
john-okeefe 078c4b1f3f docs(api): add system/hash-conflict sections to API references
Monolithic api-reference.md:
- New 'System Settings & Configuration' and 'Hash Conflicts' sections
  (endpoints, examples, response shapes) with TOC entries
- Device Management: add the sidecar config/download endpoints
- Fix stale registration flow: correct auth_url path, drop phantom
  device_id, add poll_interval/setup_instructions, status endpoint is
  POST /api/devices/register/status, and sync_endpoints point at
  /api/sync/koreader/*
- Mark PUT /api/libraries/scan-settings as legacy/superseded
- Repair Additional Resources and Collections links (dead
  COLLECTIONS_API.md / KOBO*_SETUP.md / missing-guide references)

Split api-reference.md index:
- Quick links and sections for System (settings + config) and the
  admin hash-conflict endpoints; device sidecar endpoints under Device
  Management; browse + legacy scan-settings routes under Libraries
2026-08-20 14:41:05 -04:00
john-okeefe c49a9605ff docs(api): rewrite KOReader bookmark sync for current protocol
sync_bookmarks.md documented a request shape the handler never accepted.

- Document the real body: book_uuid/book_sha256 (either required,
  SHA-256 is format-aware), plus separate bookmarks/notes/highlights
  arrays using the shared KOReader annotation shape (pos0/pos1, page,
  text, type, per-annotation book_sha256, dedup_key, percentage)
- Document color semantics from 178fb2e/dafcadd: KOReader palette
  names map to web hex swatches at the boundary, echoes carry no color
  so stored web colors survive round-trips, explicit colors are device
  edits
- koreader-protocol.md: cross-link the bookmark shape/color/dedup
  rules from the progress-sync field table
2026-08-20 14:40:59 -04:00
john-okeefe df90938c5c docs(api): document device sidecar endpoints; fix registration docs
- New get_sidecar_config.md for GET /api/devices/:id/sidecar and
  /sidecar/download: the .bookhoard.json config served to devices
  (endpoints, books keyed by per-format SHA-256 with UUID fallback,
  collections, format availability) used by the KOReader plugin to
  self-configure
- register_device.md: correct the response — no device_id at
  registration; auth_url is /devices/approve/:id (was the nonexistent
  /devices/auth/confirm/:id); document poll_interval and
  setup_instructions, and the approve-then-poll flow
- get_devices.md: fix the status endpoint path to
  POST /api/devices/register/status (was /api/devices/auth/status)
2026-08-20 14:40:54 -04:00
john-okeefe ffcdab36a0 docs(api): document hash-conflict resolution endpoints
Cover the admin API added in 03cb4c7 for duplicate-content decisions:

- GET /api/admin/hash-conflicts — pending conflict groups with member
  items and per-item usage counts (progress, highlights, bookmarks,
  notes, collections)
- POST /api/admin/hash-conflicts/:id/resolve — action=keep (merge child
  rows into keep_uuid, delete losers) vs action=keep_all (dismiss);
  JSON and form-encoded bodies, error codes including 409 for already
  resolved
- When conflicts are created (startup backfill, rescans) and the
  guarantee that files on disk are never deleted
2026-08-20 14:40:50 -04:00
john-okeefe 44a0f8c7a4 docs(api): document unified tunable system settings endpoints
The scattered scan-settings JSON routes are superseded by the new
admin-only /api/system/settings pair backed by the SettingsRegistry
(introduced in 885f6d8 / bc47450).

- Rewrite system/settings.md around GET/PUT /api/system/settings:
  SettingEntry metadata shape (type, min/max, requires_restart,
  category, group, is_default), type-aware validation rules, and the
  full tunable-setting catalog (scanner, general, security, api, sync,
  performance) with defaults, ranges, and restart requirements
- Note the legacy /api/libraries/scan-settings routes as back-compat
  only (they now refresh the registry cache on write)
- Add system/config.md for GET/PUT /api/system/config: raw key/value
  system configuration (e.g. base_url), including validation notes and
  guidance to prefer the typed settings endpoint for registry keys
2026-08-20 14:40:45 -04:00
john-okeefe 8ca95db08a docs(user): update admin and collections instructions for sidebar UI
Admin:
- Admin pages live in the sidebar's Administration panel (Dashboard,
  Libraries, Hash Conflicts, Users, Settings)
- Library creation is the Create Library modal (Library Name,
  Description, Library Type); folders are added afterwards by expanding
  the library row and using the Folders section's path input + Browse +
  Add — the old Add Library modal with folder and 'Scan on save' fields
  no longer exists
- Scanning is via the Scanner API or watch mode (File Watcher status on
  the admin dashboard); remove references to the removed per-library
  Rescan button and 'Force Rescan' option

Collections:
- Fill in the empty creating/managing placeholders with the real flow:
  New Collection button, modal fields (name, description, icon grid,
  color swatches), per-collection edit/delete icon buttons, Restore
  System button, and dashboard-section visibility via Customize
  Dashboard
2026-08-20 14:30:10 -04:00
john-okeefe 9a60196f1d docs(user): update dashboard and bookshelf instructions for sidebar UI
Dashboard:
- Customize Dashboard is opened from the icon button at the right end
  of the Library bar (next to Refresh), and requires a specific library
  selected rather than 'All Libraries'
- Library switching uses the Library dropdown in the bar below the top
  bar (includes 'All Libraries' with counts)
- Collection sections are shown/hidden from the Customize Dashboard
  toggles — the per-collection 'Show on Dashboard' setting is gone
- Mention the hover chevrons for scrolling carousels

Bookshelf:
- Saved filters: document the new toolbar buttons — Filters (opens the
  filter drawer with Apply Filters, Esc, and overlay-click close), Save,
  Load (Saved Filters dropdown with trash-icon delete), and Clear —
  replacing the emoji-labelled Save Filter / Saved Filters buttons
- Tag filtering: filters now live behind the Filters drawer on the All
  Books page
2026-08-20 14:30:05 -04:00
john-okeefe 7274b5196c docs(user): update appearance/profile instructions for sidebar UI
- Theme switching lives in the sidebar's Appearance panel (palette
  icon): swatch list with a checkmark on the active theme, and the
  'Bookshelf' section below it for wood textures; on small screens the
  sidebar opens via the top-bar menu button
- Profile: account menu is the username accordion at the bottom of the
  sidebar (not top-right); save button is 'Save Changes'
- Remove Wood Light/Dark/Mahogany from the Available Themes list (they
  are bookshelf backgrounds, not color themes) and consolidate the
  Catppuccin variants
2026-08-20 14:30:00 -04:00
john-okeefe 868003331c docs(user): update device and sync navigation for sidebar UI
The new-ui redesign replaced the top header with a sidebar and removed
the Settings pages.

- Replace all 'Settings → Devices' paths with the Devices page in the
  sidebar
- Conflicts are resolved from the book detail page's Sync Progress
  button or the Conflicts page (/conflicts); drop the nonexistent
  'Settings → Conflicts' path
- Queue status and unlinked-book references no longer invent per-device
  button paths that don't exist on the Devices page
- Reading history now points to the Progress page / book detail
- Export FAQ no longer references a Settings → Export flow that isn't
  in the UI
2026-08-20 14:29:54 -04:00
john-okeefe e310fa6a9d docs(index): update device guidance and fix broken links
- Device setup and quick-find entries now lead with the KOReader guide
  and label native Kobo sync as coming soon
- Kobo protocol/API listings tagged as a coming-soon feature
- Fix seven pre-existing broken links: contributing/Development.md had
  the wrong case (development.md), and PROJECT_GUIDELINES.md links were
  missing the ../ prefix to reach the repo root
- Refresh last-updated stamp
2026-08-20 14:14:55 -04:00
john-okeefe e35d394736 docs(readme): update device support matrix to current state
- Supported-devices table: Kobo moves from 'full support' to 'coming
  soon, use KOReader on Kobo today'; mobile apps 'coming later' with no
  speculative date
- Universal-sync pitch now states what actually syncs (position,
  bookmarks, highlights, notes) between KOReader and the web
- Frame KEPUB conversion and collection shelf mappings as groundwork
  for upcoming native Kobo support
- Fix dead links: docs/DEVELOPMENT.md → docs/developer/development.md
  and docs/contributing/DEVELOPMENT.md → actual path
2026-08-20 14:14:52 -04:00
john-okeefe 2a7ac881fd docs(user): align sync and user guides with current device support
- sync-guide: only Web and KOReader are fully supported; move Kobo to
  coming soon, drop fake Q2-Q4 2026 release dates for mobile/Kindle/
  Remarkable, and describe the plugin + server-approval registration
  flow instead of QR-code/URL approval
- sync-guide: remove cellular/mobile-app advice from battery and
  best-practice sections, correct the Calibre compatibility FAQ, update
  the changelog to reflect shipped vs. pending sync features, and fix
  the license header (GPL-3.0, not MIT)
- user-guide: lead device setup with KOReader; mark the Kobo guide as
  coming soon
- calibre-integration: OPDS client list no longer implies native Kobo
  support
- auth overview: label the mobile-application token guidance as
  'coming later' since no mobile apps exist yet
2026-08-20 14:14:49 -04:00
john-okeefe 26f1f98736 docs(kobo): mark native Kobo sync as coming soon
Native Kobo sync is implemented server-side but not yet supported on
real devices, so stop documenting it as a working feature.

- Rewrite kobo-setup.md as a coming-soon stub: point users to KOReader
  (which runs on Kobo hardware) as the supported path today, and list
  what native sync will deliver when released
- Add 'Coming Soon' status banners to the Kobo protocol spec, all five
  Kobo endpoint docs, and both API references, noting the endpoints are
  under active development and may change
- Tag the device shelf endpoints as pending native Kobo support
2026-08-20 14:14:43 -04:00
john-okeefe 54d0550dec docs(koreader): rewrite setup guide for plugin + server-side approval flow
Replace the outdated Calibre-wireless/Basic-Auth instructions with the
actual current flow: install the bookhoard.koplugin plugin, enter the
server URL in the plugin menu, then approve the pending registration
from Settings → Devices. Registration tokens are delivered to the
plugin automatically after approval (5-minute expiry), so no
credentials are ever typed on the device.

Also document bidirectional sync of position, bookmarks, highlights
(colors mapped between web and KOReader palettes), and notes, plus
format-aware SHA-256 book matching, OPDS delivery, and trimmed
troubleshooting sections covering the new registration flow.
2026-08-20 14:14:39 -04:00
john-okeefe 995ccb50bb Merge branch 'test-fixture-cleanup': book-agnostic CFI converter tests
Release / build-and-push (push) Successful in 2m36s
2026-08-20 09:22:10 -04:00
john-okeefe 4ab947f7db test(sync): replace book-specific CFI converter fixtures with a synthetic EPUB
Six converter tests pointed at absolute paths for 1984 and Crime and
Punishment under uploads/ — books that don't exist on most checkouts
(CI included), so the suite shipped with 5 permanently failing tests
(and a sixth passing only by accident: the percentage-fallback path
triggered by the missing file is the outcome it asserts).

A writeTestEPUB helper now builds a minimal deterministic EPUB in
t.TempDir() (zip → container.xml → OPF → 6-doc spine), so the tests
exercise the real zip/OPF/spine/document pipeline with no external
dependencies. The xpointer→CFI conversion, fragment-ID conversion,
both round-trips (bare and context-text-anchored), and the text-search
and percentage fallbacks all keep their original assertions, now
against known document content. internal/sync is green for the first
time on this machine.
2026-08-20 09:22:10 -04:00
john-okeefe f07c93e582 Merge branch 'sync-annotations-fix': bidirectional annotation sync for KOReader
Server-side (8 commits): web annotations finally reach KOReader and
vice versa. Fixed the 400 bind failures on every annotation-carrying
push (loose client types), resolved device-native pos0 locators for
every source (device xpointers pass through round-trip identical,
web CFIs convert to CRE xpointers with text-search anchoring, PDF
anchors map to pages), derived degenerate range ends from selection
length, echo-deduplication via served dedup keys (pull→push cycles
converge instead of minting duplicates), web↔device color mapping at
both boundaries with echo suppression (web colors flow to devices,
round-trips never drift them, device edits win), drawer-based
annotation classification, and tombstone propagation that can't
cross-delete. Perf: parsed-EPUB converter cache (bounded, locked).

Plugin-side (bookhoard.koplugin @ 4ea3966): dual-model annotation
store (KOReader 2024.07+ v2 ui.annotation + legacy v1), thin-client
collection (no per-annotation CRE lookups), dedup-key identity
matching, device-default coloring for applied highlights with
datetime_updated-based echo suppression, and native-shaped
AnnotationsModified dispatches (fixes a ReaderThumbnail crash and
paints immediately instead of after restart).
2026-08-20 09:13:24 -04:00
john-okeefe 178fb2eb37 feat(sync): serve web highlight colors to KOReader (mapped to its palette)
Reverses the earlier "no colors to the device" decision now that the
echo machinery makes it safe: GetMetadata maps the stored web hex to
KOReader's fixed color names (#ce93d8→purple, #90caf9→blue,
#a5d6a7→green, #ffd54f→yellow; pink maps to purple as the closest —
round-trip drift is prevented on the device by echo suppression, and
a device edit still wins). mapColorToKOReader restored for serving;
ingest (name→hex, preserve-on-echo) unchanged.
2026-08-20 08:43:21 -04:00
john-okeefe dafcadd211 fix(sync): echo dedup + color semantics + classification for KOReader round-trips
Echo duplication: devices push their full annotation list on every
sync, and an echo of a web-created annotation computed a different
dedup key than the original (device locators differ from web locators)
— every pull→push cycle minted a duplicate row, and cleaning those up
on the web tombstoned them back to the device, deleting the
just-applied copies. That was the "web highlights never appear on
KOReader" experience. GetMetadata now serves each annotation's
dedup_key; the device stores it on the applied entry and echoes it in
pushes; SaveHighlight/SaveBookmark/SaveNote accept a DedupKey
override so echoes converge onto the original row (verified: pull →
echo push creates no rows, LWW skips identical content).

Color semantics (per user preference): devices render their own
default and cannot round-trip web colors, so GetMetadata no longer
serves colors at all — every highlight syncs regardless of its web
color and the device draws its default. An echo carries no color;
ingest then PRESERVES the stored web color (existingHighlightColor
lookup by dedup key) so round-trips never change it. A non-empty
device color means the user edited the highlight there: it maps
name→hex (green→#a5d6a7, default yellow) and wins. Verified: echo
kept #ffd54f; a simulated device edit with "green" updated the web
row to #a5d6a7.

Classification: KOReader auto-fills text="in Chapter X" on page
bookmarks (ReaderAnnotation:updateItemByXPointer), so the plugin's
text-presence classification turned every echoed bookmark into a junk
highlight on the web. v2 classification now keys off the drawer field
(present = highlight/note, absent = bookmark with its label in note).
2026-08-19 19:41:58 -04:00
john-okeefe 50ec2bebf2 fix(reader): render device-synced highlights — synthesize range CFIs
Device-synced highlights stored POINT CFIs (epubcfi(.../8/1:1)); the
overlayer resolves those to a collapsed range and paints nothing, so
KOReader-made highlights were listed in the drawer but invisible on
the page. mapHighlightRow now builds a renderCfi: a proper RANGE CFI
(epubcfi(base,/start,/end)) synthesized from the stored start/end
points. It also repairs stale rows: missing ends (old web highlights)
and degenerate document-start ends (the old converter fallback) are
derived from the start offset plus the selection text's UTF-16
length. All overlay drawing, navigation (showAnnotation), and the
post-create/post-edit re-adds use renderCfi. Verified in-browser
against live device-synced rows: the paginator's overlayer paints
the highlight rects after the fix.
2026-08-19 14:08:03 -04:00
john-okeefe 6e9b3528d8 fix(sync): synced highlights painted nowhere — degenerate range ends + color model mismatch
Both directions synced data but rendered nothing:

- Web reader <- devices: highlights painted no overlay. Device pushes
  resolve their start xpointer exactly (text-search anchored by the
  selection) but the end conversion carries no context and fell back
  to a document-start CFI (epubcfi .../1:0) — a garbage range end.
  When the start resolved exactly, the end is now derived from it:
  same node, character offset advanced by the selection's UTF-16
  length (extendCFIByLength). Same repair when SERVING to devices,
  where old web highlights (no end anchor) and converted range CFIs
  both collapsed pos1 onto pos0 (extendXPointerByLength on the
  xpointer form) — KOReader drew zero-width highlights.
- Colors: KOReader paints from a fixed name set (Blitbuffer
  HIGHLIGHT_COLORS), the web uses hex swatches; neither understood
  the other, so device colors fell back to defaults and web hex drew
  nothing useful on devices. Both boundaries now translate: ingest
  maps names to hex (default #ffd54f), GetMetadata maps hex to names
  (default yellow) — per-datatype edits re-push with the editing
  side's color, which LWW then propagates. SyncBookmarks endpoint
  aligned to the same mapping and default.
2026-08-19 14:08:03 -04:00
john-okeefe 97e546b2a4 feat(reader): send end-anchor CFI for EPUB highlights
Web highlights stored only epubcfi_start, so devices received
degenerate pos0 == pos1 (zero-length) highlight ranges. The reader
now collapses the selection range to its end point for a second CFI
and stores it as epubcfi_end (PDF rect anchors reuse the JSON anchor
for both ends).
2026-08-18 19:13:51 -04:00
john-okeefe 1585aa1073 perf(sync): share parsed EPUBs across conversions, make converters concurrency-safe
ConvertToCanonical/ConvertFromCanonical built a fresh CFIConverter
per call, and each annotation converts twice (pos0+pos1) — a book
with 200 highlights re-opened and re-parsed the EPUB 400+ times per
sync, and again per metadata pull. A bounded 8-entry cache keyed by
path now shares converters (the parsing work belongs on the server;
clients stay thin). CFIConverter gained a mutex around its lazily
built spine/doc caches since instances are now shared between
concurrent requests.

Adds CFIConverter.SectionPercentage: book-wide percentage for a CRE
xpointer from the spine char distribution (midpoint of its document)
— the server-side counterpart to dropping per-annotation
getPageFromXPointer lookups from the plugin.
2026-08-18 19:13:51 -04:00
john-okeefe f6e257e497 fix(sync): web annotations never reached KOReader — bind 400s + unresolvable locators
Two blockers, diagnosed by simulating the plugin against the live
server with real library books:

1. Every KOReader progress push carrying annotations failed the JSON
   bind with 400 ('cannot unmarshal string into ... chapter/page of
   type int') — the plugin sends chapter:'', page:'30', and for CRE
   documents page:'/body/...' — so annotation sync AND progress sync
   failed together. KOReader annotation chapter/page now use FlexInt,
   which accepts numbers, numeric strings, empty strings, and
   non-numeric strings (decoding to 0). The server is deliberately
   liberal here so thin clients can send raw bookmark data.

2. GetMetadata served locators KOReader cannot place, so pulled items
   were junk: web bookmarks leaked 'cfi:epubcfi(...)' positions, web
   PDF highlights had empty pos0 (skipped by the plugin, invisible),
   and web deletions carried no pos0 so tombstones never matched.
   New koreaderPos0 resolver handles every source: device-native
   xpointers pass through untouched (round-trip identical, verified),
   web PDF JSON anchors map to their page number, EPUB CFIs convert
   to CRE xpointers (selection text passed as text-search context for
   exact anchoring), 'page:N' positions strip to the bare number.
   Unresolvable annotations are skipped with a log line instead of
   poisoning devices; tombstones get pos0 injected from the new
   locator columns.

Also: thin clients omit per-annotation percentages (paging docs still
send arithmetic page/total); the server derives them — section
midpoint from the spine char distribution for CRE documents, page/
page-count for fixed formats.
2026-08-18 19:13:38 -04:00
john-okeefe 0670d904a0 feat(db): locator columns for tombstoned annotations
GetTombstonedAnnotationsForBook now also returns each tombstone's
start_position/end_position and epubcfi_start/end (note: position/
epubcfi_location, bookmark: position/cfi_position), so serving code
can resolve a device-native locator for deletions of web-created
annotations, whose device_sync_data carries no pos0.
2026-08-18 19:13:23 -04:00
john-okeefe 922336c064 Merge branch 'reader-redesign': reader v2 — immersive chrome, annotations, search, touch, webtoon
Release / build-and-push (push) Successful in 3m0s
Full reader redesign across 22 commits (with the foliate-js fork's
zoom-control engine work pinned per release):

- Phase 0: panel/chrome stabilization, bookmarks end-to-end (REST CRUD
  via AnnotationService), dead UI removal, tombstone resurrection fix
- Phase 1: edge-to-edge glass chrome with auto-hide, slide-over drawers,
  tri-state PDF pointer mode (Smart/Pan/Text), Kindle-style theme swatches
- Phase 2: touch gesture engine (pinch/pan/swipe/double-tap), tap zones,
  mobile sheets + compact toolbar with overflow menu
- Phase 3: EPUB highlights & notes (selection popover, overlayer
  rendering, annotations drawer), PDF text highlights (fraction-rect
  overlays), in-book search for both EPUB and PDF, back-to-location
  stack, page thumbnails, shortcuts help modal, desktop edge zones
- Phase 4: webtoon (vertical-scroll) mode for comics, brightness/
  contrast/night filters, bookmark toast feedback
- Build hygiene: vite stale-chunk cleanup, browser-verified fixes for
  Alpine proxy/dpr/duplicate-key classes of bugs along the way
2026-08-18 09:53:10 -04:00
john-okeefe 243d369d21 fix(reader): pin foliate-js e448d36 — webtoon pages now load
The initial webtoon commit's IntersectionObserver (shadow-host root)
never delivered intersections in Chromium, leaving pages blank.
Scroll-driven loading in e448d36 fixes it; verified end-to-end in a
real browser: pages render (content-rich screenshots), deep scroll
advances the reading position (7/10) and progress readout, filters
visibly change both webtoon images and PDF pages via ::part(filter)
(brightness 5% -> 57% smaller screenshot), paged comics still use
foliate-fxl, and webtoon UI gating (zoom/spread hidden) works.
2026-08-18 08:34:44 -04:00
john-okeefe e500039d1b feat(reader): webtoon reading mode + brightness/contrast/night filters
Phase 4 of the reader redesign (foliate-js ea268df):

- Webtoon mode for comics: continuous vertical scroll of all pages
  (900px centered column on wide screens), lazy-loaded with a 150%
  IntersectionObserver margin, far pages unloaded to bound memory
  with stable aspect-ratio placeholders so the scrollbar never jumps.
  Chosen per book (Paged | Webtoon segmented control in Settings →
  Layout & Display; stored in localStorage per media item since a
  webtoon title and a paged manga volume want different flows).
  Toggling reloads the reader — the renderer is chosen at open time —
  and progress restores from the saved page. Relocate events flow
  through the same pipeline, so the slider, progress saving, back
  stack, tap zones, and edge zones all work unchanged. Zoom/fit/
  magnifier/spread controls hide in webtoon (natural-width scroll).
- Display filters for fixed-layout: brightness (30-130%) and
  contrast (70-130%) sliders with live preview, plus Night Mode
  (invert) — also a quick row in the ⋯ tools menu. One --fx-filter
  CSS var drives everything: ::part(filter) on foliate-view iframes
  (forwarded via the new exportparts attribute) and the webtoon
  page images alike. Persisted as fx_brightness/fx_contrast/fx_invert
  (types + defaults both sides); Restore Defaults resets them.
2026-08-18 08:25:00 -04:00
john-okeefe 94be6edceb feat(reader): shortcuts help modal + desktop edge page-turn zones
Help menu (the reader had a growing shortcut/gesture vocabulary with
no discoverability): a ? topbar button, the '?' key, and F1 open a
glass modal listing navigation, zoom/pan, highlight, and touch
gesture reference — format-aware (fixed-layout/PDF rows appear only
where they apply), Esc closes it first in the dismiss chain.

Desktop edge zones: clickable page-turn strips on the left/right
viewport edges (8% width, 44-72px), desktop only (hover+fine-pointer
media query — touch devices use tap zones, avoiding double paging).
Hovering reveals a chevron arrow and a subtle edge gradient. Zones
disable (pointer-events pass-through) while a fixed-layout page is
zoomed so edge clicks belong to content: panning, selection,
highlight editing. fxZoomed tracks zoom state via the renderer zoom
event, reset/fit actions, and init.
2026-08-18 07:59:26 -04:00
john-okeefe 1a07635605 feat(reader): confirm bookmark creation with a success toast
The 🏷️ bookmark button (and the 'b' shortcut) saved silently — an
accidental click gave no reaction at all. addBookmark() now shows a
short success toast ('Bookmark added — <progress>') using the
existing toast system, which the reader bundle hadn't been importing.
Importing it also activates the shared fetch interceptor, so failed
reader API calls (incl. bookmark saves) surface error toasts instead
of being swallowed.
2026-08-17 14:28:09 -04:00
john-okeefe 206db93587 fix(reader): PDF contents drawer rendered nothing — duplicate x-for keys
Diagnosed in a real browser (playwright/chromium against the running
app + Head First SQL): the engine's book.toc held all 18 entries with
correct labels/hrefs and the tab counter even showed 380, yet zero
links rendered while the console flooded with 'Alpine Warning:
Duplicate key on x-for'.

Root cause: the drawer keyed TOC rows by item.href. PDF outline
entries frequently share the same destination (e.g. the printed TOC
page is targeted by several bookmark entries), so flattened items
carried duplicate keys — and Alpine's x-for renders NOTHING for a
duplicated key, not even the unique ones. EPUB TOCs never collided
because their hrefs are unique file paths, which is why this only
surfaced on PDFs.

Key is now href + row index (the list is static once loaded, so
positional keys are safe). Verified end-to-end in the browser: 18
entries render and the drawer populates.
2026-08-17 14:14:33 -04:00
john-okeefe fd4c357d39 feat(reader): page thumbnails tab + reliable PDF contents
Investigation: the contents drawer read book.toc, which makePDF
builds from pdf.getOutline() — verified against the real library PDF
(Head First SQL) through the exact vendored pdf.js build AND the exact
range transport the browser uses: 18 chapter entries come back. So
the source is right; manga-scan PDFs and CBZs simply have no embedded
outline, which made Contents look broken exactly where users expect
page-based navigation.

- TOC now populates eagerly right after the book opens (toggle-time
  lazy population removed), so an existing outline can never silently
  miss due to timing; the drawer keeps the honest empty-state text
  for books without outlines.
- New 'Pages' tab in the contents drawer for fixed-layout books:
  a Kavita-style thumbnail grid (3-up, current page highlighted and
  scrolled into view, click to jump — recorded on the back-to-
  location stack). Thumbnails render client-side: PDFs via the
  in-memory pdf.js document (small viewport render, Alpine.raw
  unwrap); comics via the page's image blob drawn down to a 110px
  canvas, then unloading the full-size blob so thumbnailling doesn't
  hoard page images. Lazy via IntersectionObserver scoped to the
  drawer's scroll container (200px margin), canvases cached at module
  level so revisits are instant; failures warn in console and allow
  retry. The backend /readers/thumbnails endpoint turned out to be an
  empty stub, so nothing server-side was worth wiring.
2026-08-17 13:55:21 -04:00
john-okeefe dc68d03360 fix(reader): both toolbars showed below 768px — cascade-layer conflict
The display:none for the full toolbar lived in @layer components while
the div also carried Tailwind's flex utility (@layer utilities). Layer
order beats specificity, so the utilities layer always won and the
full bar never hid below the breakpoint (the compact row only worked
because it had no display utility of its own).

Switch to Tailwind's own responsive utilities in the markup — full
toolbar 'hidden md:flex', compact row 'flex md:hidden' — and delete
the custom rules; responsive display now resolves inside a single
layer where source order (responsive variants after base) guarantees
the right winner.
2026-08-17 13:42:02 -04:00
john-okeefe 6cd0fb226a feat(reader): mobile-pattern toolbar — compact row + ⋯ overflow menu under 768px
Wrapping alone isn't how polished mobile readers work. Adopt the
standard pattern (Kindle/Apple Books/Mihon) responsively:

- >= 768px: the full fixed-layout toolbar stays (wrap still absorbs
  mid-size widths) — power users keep one-click zoom/fit/spread.
- < 768px: single-line compact row — page back, back-to-location pin,
  slider, page forward, progress, and a ⋯ overflow button. No
  wrapping, no horizontal scroll.
- ⋯ opens a glass menu anchored above the bar with LABELED rows
  (Zoom −/%/+, Fit, Page position/Recenter, Magnifier, Pointer
  Smart/Pan/Text, Double page, Contents) — labels beat mystery icons
  on touch. Pointer row hides for comics; menu scrolls if tall.
- Dismissal: Esc, outside click (⋯ button exempt so it re-toggles
  cleanly), opening any drawer or TOC closes it; hides with the
  chrome. Compact slider registered in progressSliders() so all
  three stay in sync with relocate events.
2026-08-17 13:35:14 -04:00
john-okeefe f283903e2b feat(reader): relocate back-to-location, add recenter control, wrap bottom bar on small windows
- Back-to-location moves from the topbar (where it sat between Back
  and the title, too subtle and disconnected from navigation) into
  both bottom-bar rows, beside the page-back arrow — the natural
  'go back' cluster. New icon: a location pin, clearly distinct from
  the back arrow and page controls. Appears only when the stack has
  a return target; Alt+← unchanged.
- New recenter button in the fixed-layout row (crosshair icon, next
  to zoom): resets pan offsets while keeping the current zoom —
  backed by foliate's new recenter() (1c812e8), which zeroes the
  wrapper translate and re-syncs the spread side.
- Both bottom-bar rows wrap gracefully on narrow windows instead of
  overflowing/h-scrolling: controls are grouped (paging+back | slider |
  fit+zoom+magnifier+recenter | pointer mode | spread | progress+TOC)
  so groups flow to a second line at small widths; the slider shrinks
  first (grow + min-width), everything else stays whole. Fixed-layout
  row drops its overflow-x-auto.
2026-08-17 13:23:59 -04:00
john-okeefe 34a27a5951 fix(reader): PDF highlights offset from the words — wrong fraction denominator
Highlights landed on the right line but shifted right and oversized
on any display with devicePixelRatio != 1. Cause: selection fractions
divided the textLayer span rects by documentElement's screen rect,
but pdf.js scales the iframe's <html> by 1/dpr — that rect is dpr×
smaller than the visible page, inflating every x/w fraction by dpr
(on a 2× display a highlight started twice as far right and was twice
as wide). dpr=1 displays were coincidentally correct, which is why
the geometry looked sound when written.

The denominator is now the rendered canvas (#canvas canvas), whose
post-transform rect IS the visible page and shares the textLayer's
transform space — the dpr scaling cancels exactly. Comics keep the
img denominator; a viewport fallback covers any page without either.
The popover-placement scale factors (frame/denominator) become 1 for
PDFs as a side effect, fixing popover drift too. The fork's click
hit-test (86e234d) gets the same canvas-aware denominator so clicking
highlights opens the editor at the right spot.

Highlights saved before this fix stored dpr-inflated fractions and
will still render misplaced — delete and re-create them.
2026-08-17 08:33:11 -04:00
john-okeefe 1905feceea fix(reader): PDF search returned nothing — reactive proxy broke pdf.js; add back-to-location
PDF search diagnosis: extraction and matching were proven correct
against the real 609-page library PDF (pdfjs 5.5.207, incl. the exact
range-transport setup makePDF uses — 841 hits for 'SELECT'), and the
served bundle had every piece. The failure was Alpine's reactivity:
this.book is a plain object, so reading .pdf through component state
returns a reactive Proxy around the PDFDocumentProxy — and pdf.js
v5 uses #private fields, so getPage() through the proxy throws
'cannot read private member', which the empty catch rendered as a
silent empty result set. runPdfSearch now unwraps via Alpine.raw
(falls back to the raw read), and search failures surface in the
drawer ('Search failed — see console') plus console.warn instead of
masquerading as 'No matches'.

Back-to-location stack (research/footnote workflow): the current
position is recorded before every programmatic jump — search-result
clicks, TOC entries, bookmark and highlight jumps — and on every
internal link click (footnotes, cross-references) via foliate's
'link' event. A ↩ button appears in the topbar once a return target
exists; Alt+← works everywhere. Ordinary paging never pollutes the
stack (max depth 50, consecutive duplicates collapse).
2026-08-17 08:23:57 -04:00
john-okeefe 6fc4107e3c feat(reader): in-book search for PDFs
PDFs have fully searchable text (pdf.js text layer) — the previous
reflowable-only gate existed only because foliate's generic search
needs DOM documents that PDF sections don't provide. This adds a PDF
pipeline alongside it:

- Fork d065495 exposes the pdf.js document proxy as book.pdf so the
  host can drive text extraction directly.
- New web/src/reader/pdf-search.ts: extractPdfPages() pulls each
  page's textContent with item geometry (progress-reported, cached
  after first search). PDF text items often omit inter-word spaces
  (gaps are positional), so pages are joined gap-aware — baseline
  changes, hasEOL, or horizontal gaps past a font-size threshold
  become spaces — recording a char→item map. searchPdfPages() does
  case-insensitive matching over the joined text and maps each hit
  back to the page-fraction rects of the items it spans, with
  ellipsized pre/match/post excerpts. Pure functions, unit-sanity
  checked (cross-item 'brave new' → two rects).
- runSearch branches: EPUB keeps foliate's DOM search; PDFs search
  the extracted pages, group hits per page ('Page 12'), and render
  on-page hit rectangles through the existing fraction-rect overlay
  (addRectAnnotation) — which re-render automatically when pages
  revisit, same as highlights. Clearing the query removes them.
- Results navigate by page index; the 🔍 button and '/' shortcut now
  appear for PDFs too (comics remain without searchable text).
2026-08-17 08:04:30 -04:00
john-okeefe 5e73b0a4f6 feat(reader): in-book search for reflowable formats
Wires foliate's search engine into the new drawer system:

- 🔍 topbar button (reflowable-only; PDF/comic sections have no
  searchable text documents) and the '/' keyboard shortcut open a
  Search drawer: query input (Enter to run), live progress while
  scanning (per-section percent), match count, and results grouped
  by section with TOC labels.
- Each result shows pre/match/post excerpt rendered as three text
  nodes (no x-html — book content never enters the DOM as markup);
  the match is styled with a translucent <mark>. Clicking jumps to
  the hit's CFI and closes the drawer.
- Hits are drawn on the page through foliate's overlayer (outline
  style) and persist across page turns — the engine re-applies
  search results when a section's overlay is created. Clearing the
  query removes the outlines.
- A generation counter discards results and progress from superseded
  searches (rapid re-query), and starting a new search clears the
  previous one server-side via view.clearSearch().
- Search integrates with the drawer system: scrim, Esc-to-close,
  one-drawer-at-a-time, / focuses the input via .
2026-08-17 07:58:03 -04:00
john-okeefe eb09a5d939 fix(reader): PDF highlights never appeared — isPDF read too early + overlay shrunk by pdf.js transform
Two bugs broke the Phase 3b PDF highlight flow end to end:

1. Selection capture never attached: reader.ts read renderer.isPDF
   before view.init() rendered the first spread, but the renderer
   only sets that flag once frames exist (PDF frames carry pdf.js
   onZoom). The stale undefined copy gated the pointerup selection
   listener off, so selecting PDF text did nothing. The listener now
   gates structurally on the loaded document having a .textLayer
   (true for every PDF page, false for comics), and isPDF is re-read
   after init — which also finally makes the Smart|Pan|Text control
   and the saved pointer mode apply on PDFs.

2. Highlights rendered invisibly: the overlay SVG lived inside the
   page iframe, whose <html> pdf.js scales by 1/devicePixelRatio —
   shrinking the overlay into the top-left corner on any dpr != 1
   display. The fork (1c0ebf3) now renders annotation rects
   host-side, inside the frame wrapper element, positioned in
   percentages of the visible page box — immune to the html
   transform, zoom re-renders, comic iframe scaling, and pan/zoom.
2026-08-17 07:47:03 -04:00
john-okeefe a05b0167ad feat(reader): PDF text highlights via fraction-rect annotations
Phase 3b of the reader redesign — highlighting for fixed-layout PDFs:

- Select text on a PDF page → same glass popover as EPUBs (colors,
  note, copy). The selection's client rects are normalized to
  page-fraction quads using a transform-inclusive denominator so
  pdf.js's devicePixelRatio scaling on <html> cancels out, then
  stored as a JSON anchor {page, rects} in epubcfi_start.
- Rendering goes through the fork's new rect-annotation pipeline
  (foliate-js aba68d8): a full-bleed viewBox-0-100 SVG inside the
  page iframe, so highlights stay aligned through pan/zoom, iframe
  CSS-scaling, and PDF hi-res re-renders with zero re-anchoring.
  Frames carry their page index and re-render annotations when
  recreated on spread changes.
- Clicking an existing highlight hit-tests in fraction space and
  opens the edit popover (recolor, note, copy, delete) at the
  host-space click position; drag-selecting text never triggers it.
- Annotations drawer: PDF highlights jump by page index; notes and
  recolors round-trip through the same LWW/dedup sync path as EPUBs
  (same dedup key derivation on the JSON anchor).
- Comics keep bookmark-only highlighting (no text layer) by design.
2026-08-16 12:48:09 -04:00
john-okeefe 40d70513da feat(reader): EPUB highlights & notes — selection popover, overlayer rendering, annotations drawer
Phase 3 (EPUB half) of the reader redesign:

- Select text in a reflowable book → floating glass popover at the
  selection (5 colors, note, copy). Clicking a color creates the
  highlight via POST /api/media-items/:id/highlights, anchored by the
  foliate range CFI (epubcfi_start) with percentage position.
- Highlights render through foliate's overlayer pipeline: draw-
  annotation draws Overlayer.highlight with the stored color,
  create-overlay re-adds persisted highlights as sections load,
  show-annotation opens the edit popover when a highlight is clicked
  (recolor, edit note, copy, delete).
- Backend: highlight create/update accept epubcfi_start/end,
  note_text, and percentage fields; position validation relaxed
  (CFIs exceed the old 100-char cap); PUT routes through
  AnnotationService.SaveHighlight so edits get dedup/LWW treatment
  and actually persist note_text (the plain query can't).
- Bookmarks drawer becomes the Annotations drawer with tabs:
  Highlights (color-bar list, note previews, jump/edit/delete),
  Notes (add note at current position, list, delete — backed by the
  existing notes API), and Bookmarks (unchanged behavior).
- Popover dismissed on outside click, collapsed selection, page
  navigation, or Esc (new top-priority Esc branch).
2026-08-16 12:33:55 -04:00
john-okeefe bd7d71a284 build(vite): remove stale hashed chunks after each build
emptyOutDir is false because web/static also holds tracked assets,
so *-<hash>.js chunks from every previous build accumulated
indefinitely and leaked into Docker images via the build context
(the reader serves whichever chunk the import chain names, so the
orphans are pure confusion + bloat). A closeBundle plugin now
deletes any hashed chunk this build did not produce.
2026-08-14 16:06:27 -04:00
john-okeefe 24ea9d8a38 feat(reader): touch & mobile — tap zones, gesture engine, mobile sheets
Phase 2 of the reader redesign:

- Fixed-layout touch engine (foliate-js e9e61d8): pinch-zoom around
  the midpoint, two-finger pan, single-finger pan while zoomed,
  horizontal swipe page-turn at fit (RTL-aware via next()/prev()),
  and double-tap to zoom 2.5x / reset. Touch events forwarded from
  page iframes with converted coordinates; preventDefault only when
  the engine consumes the gesture, so PDF text selection and native
  taps stay intact. touch-action: none on the host and in comic/pdf
  page documents keeps the browser from fighting the engine.
- Tap zones (Kindle-style) for touch devices: tap the outer margins
  to page, center to toggle chrome. Size configurable (10-50%) via
  the revived tap_zone_size setting; toggle via new tap_zones_enabled
  (Behavior section of the settings drawer). Pointer-based + passive
  so drags/swipes/selection never trigger; attached both to the
  viewport and inside every page document (iframe events don't
  bubble); debounced 280ms so double-tap zoom doesn't also page; no
  zone actions while a fixed-layout page is zoomed.
- Drawers become full-width sheets on screens <= 640px.
2026-08-14 16:00:19 -04:00
john-okeefe a962342ee0 fix(sync): resurrect tombstoned annotations when a newer save re-creates them
Deleting a bookmark/highlight/note and then re-adding the same content
at the same position (same dedup key — e.g. the reader's auto-titled
'Bookmark at X%') was silently swallowed: the save hit the tombstone
branch, returned 201 with the deleted row, and the list (which filters
deleted) stayed empty. Bookmarks were further blocked by the
UNIQUE(media_item_id, user_id, title) slot the tombstoned row holds,
and notes had no TTL escape at all.

Tombstones now only block saves that predate them (stale replays from
a device that still has the annotation). A save whose modification
time is newer than max(deleted_at, last_modified_at) — a deliberate
re-create from the web or a device — resurrects the row via the LWW
update queries, which now clear deleted/deleted_at.
2026-08-14 15:42:18 -04:00
john-okeefe 14d1a158a0 feat(reader): glass chrome — translucent bars, custom slider, slide-away hide
Modernize the reader chrome bars without touching the drawer system:

- Bars become theme-tinted glass: 70% bg-primary translucency over
  the edge-to-edge page, 18px backdrop blur + saturation, hairline
  translucent borders, soft directional shadows (single .reader-glass
  class owns the effect; replaces solid opaque backgrounds and the
  tailwind backdrop-blur that would override it).
- Chrome hide/show now slides the bars off-screen (translateY) in
  addition to the opacity fade, via .chrome-hidden on #reader-chrome.
- Theme-aware hover pills (translucent currentColor tint) replace
  hard-coded gray-700 hovers; focus-visible rings added.
- Progress slider gets a custom thin rounded track with a floating
  white thumb (webkit + gecko), replacing native range styling.
- Separators and the fit-mode select match the glass language
  (.reader-sep, .reader-select).
2026-08-14 15:16:05 -04:00
john-okeefe 612f888683 feat(reader): immersive chrome, slide-over drawers, tri-state PDF pointer mode
Phase 1 of the reader redesign:

- Reading surface is edge-to-edge; top/bottom bars overlay
  translucently (backdrop-blur) instead of reserving insets, killing
  the inset-coordination bug class entirely. Chrome auto-hides after
  2.5s of pointer inactivity (chrome_behavior setting finally wired:
  auto-hide / always-visible; legacy values map to auto-hide). Pointer
  activity inside page iframes keeps it awake; Esc toggles.
- TOC / Settings / Bookmarks become slide-over drawers with a scrim
  (z-50, full-height, safe-area aware), replacing the dockable-panel
  system and its window-shade headers. Only one drawer opens at a
  time; Esc or scrim click closes.
- Bottom bar is contextual: reflowable keeps nav/slider/progress/TOC;
  fixed-layout row adds Fit Page/Width select, zoom cluster,
  magnifier (now shows active state), Double Page Spread toggle, and
  a Smart | Pan | Text segmented control replacing the cryptic
  two-state icon. Smart = text-aware drag; Text = selection-only
  (manual smart-detect off); Pan = force pan. Choice persists via
  pdf_interaction_mode (new setting + foliate 29bc958 'text' mode).
- Settings drawer: Behavior (chrome, progress mode), Appearance with
  18 Kindle-style theme swatches (single source of truth from
  THEME_COLORS), Typography, Layout — each scoped by format.
- Keyboard: t/s/b open TOC/settings/bookmark, Esc closes drawers
  before toggling chrome, shortcuts skip form inputs; both slider
  rows tracked correctly (no duplicate-ID lookups).
- Topbar: Back, title, add-bookmark, bookmarks drawer, Aa settings;
  chrome follows user theme.
2026-08-14 14:59:51 -04:00
john-okeefe ba95cc3e8b fix(reader): stabilize chrome panels, bookmarks end-to-end, dead UI removal
Phase 0 of the reader redesign:

- Panels no longer render under the top/bottom bars: sidebars get
  measured insets (same resize/safe-area mechanism as the viewport);
  panel max-height now derives from the bounded sidebar instead of a
  100vh guess; right-side border targets the actual sidebar.
- Bookmarks work end-to-end for the first time: REST CRUD under
  /api/media-items/:id/bookmarks (create/delete route through
  AnnotationService for dedup/LWW/tombstones), fix UpdateMediaBookmark
  referencing nonexistent updated_at column, frontend posts to the
  real API with per-format position (CFI vs page), live list with
  jump + delete instead of SSR-only snapshot.
- Fix chapter matching in progress saves: boundaries were compared by
  a nonexistent tocItem property, so chapter was never persisted.
- Remove dead UI: Navigator panel stub, empty dictionary popup shell,
  unwired Chrome Behavior select; purge 160 stale build artifacts.
- Reader chrome now follows the user's app theme instead of hardcoded
  theme-tokyo-night.
2026-08-14 09:05:33 -04:00
john-okeefe 03cb4c7869 feat(admin): startup hash backfill and hash-conflict resolution API
Release / build-and-push (push) Successful in 2m48s
Complete the SHA-256 lifecycle for preexisting databases: items
imported before hashing existed get hashed automatically, and any
content duplicates discovered in the process land on the new admin
Hash Conflicts page for an explicit keep/merge decision.

HashBackfillService (runs once 30s after startup, independent of
auto-scan):
- hashes every media_items row where file_sha256 IS NULL, resolving
  each path through LibraryService; per-item failures are logged and
  skipped so one unreadable file cannot block the pass
- no-op once everything is hashed (logged and skipped)
- finishes with a conflict sweep flagging every content-duplicate
  group via FindHashConflictGroups + CreateHashConflict; the sweep
  runs after the per-item pass because a preexisting pair only
  becomes detectable once both sides have their hash

API (admin-only):
- GET /api/admin/hash-conflicts - pending groups with member items
  and usage counts
- POST /api/admin/hash-conflicts/:id/resolve - action=keep_all, or
  action=keep with keep_uuid: validates the uuid belongs to the
  group, re-parents every other copy's child rows onto the kept item
  (reparent_media_item_children), deletes the losers, and records
  the resolution + resolving admin; accepts form or JSON bodies and
  returns the htmx resolved fragment

Page route /admin/hash-conflicts (admin-only) renders the template
with hydrated conflict data; HashConflictsHandler wired into the
router Config and constructed in main.

Verified end-to-end against the live database: duplicate detection,
pending listing, keep_all resolution, merge path (re-parent +
delete), and - critically - a resolved group is not re-flagged by a
later sweep (upsert no-op). Database restored afterward.
2026-08-14 08:53:01 -04:00
john-okeefe 8004cb81a5 feat(ui): admin Hash Conflicts page and nav entry
New /admin/hash-conflicts page (admin-only) listing pending
content-duplicate groups. Each group card shows the library, a
shortened SHA-256, and one row per copy with title, author, path,
size, and per-copy reading-data counts (progress, highlights,
bookmarks, notes, collections) - copies that own user data are
highlighted so the keep choice is informed.

Per copy: 'Keep this copy' merges the other copies' child rows into
it and deletes them. Per group: 'Keep both' for intentional
duplicates. Both confirm first, resolve via htmx POST, and swap the
card for a resolved confirmation inline. The confirmation fragment is
built inline in the handler rather than the templates package
(templates imports handlers; a back-import would be a cycle).

Empty state shown when no conflicts are pending. Adds a 'Hash
Conflicts' entry to the admin sidebar section between Libraries and
Users.
2026-08-14 08:52:33 -04:00
john-okeefe 77990d0dc0 feat(scanner): recompute hashes on rescan and flag content duplicates
Force rescan was metadata-only: updateMediaItem never touched the
hash identifiers, so a force scan could not backfill file_sha256 for
items imported before hashing existed (or where extraction originally
failed). Those items were invisible to content dedup and SHA-256
device matching with no way to fix short of delete + re-import.

processMediaFile now refreshes hash identifiers in three cases:
- force rescan (the admin Scan button becomes the backfill tool)
- file size change (stored hash is stale - the bytes changed)
- unchanged file with no stored hash (ordinary scans self-heal the
  legacy backlog incrementally, no admin action required)

Each recompute runs recordHashConflictIfAny: when the freshly stored
hash is now shared by more than one item in the library, the group is
upserted into hash_conflicts for the admin Hash Conflicts page. The
upsert is a no-op for already-tracked groups, so resolved 'keep both'
decisions stick.

Also extract a package-level computeFileSHA256 (the scanner method
now delegates to it) so the startup backfill service can hash files
without a scanner instance.
2026-08-14 08:52:18 -04:00
john-okeefe 0c39e04e4a feat(db): hash_conflicts table and backfill/conflict queries
Content duplicates (same library + file_sha256 at different paths,
e.g. the same book imported twice under two names on a preexisting
database) cannot be auto-collapsed the way path duplicates were:
keeping both copies may be intentional. Surface them for an explicit
admin decision instead.

Schema:
- new hash_conflicts table keyed (library_id, file_sha256) with a
  status/resolution lifecycle: 'pending' until an admin resolves via
  'keep_all' or 'kept:<uuid>' (which copy was kept after merging)
- resolution is VARCHAR(50) - 'kept:<uuid>' is 41 chars; include a
  widening ALTER for databases created with the initial 30-char width
- resolution/resolved_by/resolved_at record who decided what and when

Queries:
- ListMediaItemsMissingHash: items imported before hashing existed
  (file_sha256 IS NULL), ordered oldest-first for the backfill pass
- FindHashConflictGroups: the content-duplicate group detection
  (GROUP BY library_id, file_sha256 HAVING COUNT(*) > 1)
- ListMediaItemsBySHA256AndLibrary: full membership of one group
- CreateHashConflict: upsert with DO NOTHING so already-tracked groups
  are untouched - critical behavior: a group an admin resolved as
  'keep both' is never re-flagged by later sweeps
- ListPendingHashConflicts: admin listing with library name and live
  item counts (items may have been deleted since flagging)
- GetHashConflict / ResolveHashConflict: lifecycle
- GetMediaItemUsageCounts: per-item progress/highlight/bookmark/note/
  collection counts so the admin can make an informed keep choice
- ReparentMediaItemChildren: sqlc binding for the existing
  reparent_media_item_children() migration function, used to merge a
  losing copy's child rows into the kept copy
2026-08-14 08:52:05 -04:00
john-okeefe 8599e5c250 docs(sync): document SHA-256 fields and format-aware matching in koreader protocol
Bring the koreader protocol doc in line with the hash-sharing work:

- Request table: uuid is no longer required (it is absent on the first
  sync of a newly downloaded book); document sha256 and file_path and
  the resolution priority uuid -> sha256 -> file_path alias ->
  title/author
- Note that SHA-256 matching is format-aware (media_items hash first,
  media_item_formats fallback) so converted KEPUB/PDF downloads match
- Document the sha256 field returned by the metadata and library
  endpoints
- Add a 'Book identification' section pointing current and future
  clients (koreader, kobo, OPDS, device-link UI, mobile apps) at the
  shared BookResolver as the single resolution path
2026-08-14 08:26:47 -04:00
john-okeefe 830741cd65 feat(sidecar): key book map by per-format hashes
The sidecar config's books map is keyed by the primary SHA-256 (UUID
fallback). A device holding a converted format (KEPUB/PDF) whose hash
lives only in media_item_formats could not resolve its file through
the sidecar.

After inserting the primary-keyed entry, also register the same entry
under each per-format hash from media_item_formats (first write wins,
so a primary hash is never shadowed). Devices now resolve converted
files via the sidecar the same way the server's BookResolver does.
Applied to both the GET and download sidecar builders.
2026-08-14 08:26:32 -04:00
john-okeefe 48af5d3e14 feat(opds): always send X-Bookhoard-SHA256 on native EPUB downloads
DownloadBook populated fileSha256 only for the kepub and pdf format
branches, so the default EPUB download never emitted the
X-Bookhoard-SHA256 response header - the hash was only available in
the feed metadata, not on the download response itself.

Populate it from mediaItem.FileSha256 in the default branch so every
download response carries the canonical primary-format hash. Clients
that capture response headers at download time now learn the hash
regardless of which format they requested.
2026-08-14 08:26:14 -04:00
john-okeefe 5584bdefb5 feat(koreader): resolve pushes by SHA via BookResolver and return SHA on pull
Fixes the 'cannot push until pulling first' wall on books downloaded
via OPDS. Root cause chain: the bookhoard koreader plugin only learns
the book UUID from a successful push response, but the first push had
to match by SHA-256 alone - and that match consulted only
media_items.file_sha256, missing converted formats. When the hash
missed, no UUID was returned, so pull stayed blocked (it requires the
UUID) and the book could not sync at all.

Resolution side - route all five SHA-256 match sites through the shared
BookResolver so they are format-aware:
- resolveBookToMediaItem priority 2
- SyncBookmarks book-level lookup
- per-bookmark, per-note, and per-highlight override lookups

Exposure side - return the canonical hash so clients can learn and
cache it from a pull regardless of how the book was obtained:
- KOReaderMetadata gains sha256, populated from mediaItem.FileSha256
- KOReaderLibraryBook gains sha256, populated the same way, so the
  library list endpoint carries it for every book

Together with the plugin-side UUID bootstrap (bookhoard.koplugin),
push and pull now work in either order on any format.
2026-08-14 08:25:46 -04:00
john-okeefe 60a94df8e1 feat(sync): add shared BookResolver with format-aware SHA-256 matching
The platform had three duplicated, divergent book resolvers (koreader,
kobo, BookMatchingService) and none of them consulted
media_item_formats.file_sha256 - per-format hashes for converted files
(KEPUB, PDF) are computed and stored at import/conversion time but were
never used for lookup. GetMediaItemFormatBySHA256 existed with zero
callers. Any client holding a converted file could never match by
hash.

Add internal/services/book_resolver.go: a single shared resolution
path from client-supplied identifier to media_item.
ResolveBySHA256 checks media_items.file_sha256 first (indexed
GetMediaItemBySHA256), then falls back to media_item_formats.
file_sha256 (indexed GetMediaItemFormatBySHA256, first caller) so a
converted format matches with equal confidence. The import-time
SHA-256 is the canonical identifier shared by every interface.

Wire two of the existing resolvers through it:

- BookMatchingService.matchBySHA256: replaces the in-memory
  ListMediaItems scan of up to 1000 rows with the resolver's indexed
  lookups, and gains format awareness for the link/auto-link UI.
  MatchMethod now reports sha256_sha256 or sha256_sha256_format
- KoboHandler.mapContentIdToBookhoardUUID: the SHA-256 heuristic
  branch (ContentId that looks like a 64-char hash) now resolves
  format-aware too. Kobo's entitlement_id wire identity is untouched;
  only the opportunistic hash branch changed
2026-08-14 08:25:23 -04:00
john-okeefe 9b171a0060 fix(scanner): prevent duplicate media item imports
A read-then-write race in processMediaFile allowed the same file to be
imported twice: two concurrent scan jobs (startup scan, fsnotify dirty-
directory scan, periodic backup poll, or a manual scan each run on
separate worker goroutines with separate MediaScanner instances) could
both SELECT 'not found' and both INSERT. There was no transaction, no
row lock, no unique constraint on (library_id, file_path), and no
ON CONFLICT clause, so nothing stopped the double insert. Observed in
production as two identical 'Head First SQL' rows created in the same
second (same sha256, size, path, library).

Database enforcement:
- schema.sql: add UNIQUE(library_id, file_path) constraint, guarded so
  re-runs don't error
- schema.sql: add self-healing migration that runs on every startup -
  dedup_media_items_by_path() collapses existing path-duplicates and
  reparent_media_item_children() moves all child rows (progress,
  highlights, bookmarks, notes, collections, formats, aliases, kobo
  entitlements, etc.) onto a survivor before deleting losers, so the
  constraint applies cleanly on already-duplicated servers without
  losing reading history. Survivor picks the row with the most user
  data, ties broken by lowest id
- CreateMediaItem: upsert via ON CONFLICT (library_id, file_path) DO
  UPDATE so concurrent inserts collapse to one row and return it
- CreateMediaItemFormat: upsert via ON CONFLICT (media_item_id,
  format_type), closing the same race on format rows

Application-level guards:
- media_scanner processMediaFile: after computing the file hash, check
  GetMediaItemBySHA256AndLibrary (new query) and treat the file as
  existing when identical content is already in the library under a
  different path (content dedup, library-scoped so multi-library
  setups still work)

Ops tooling:
- scripts/dedup_media_items.sql: standalone idempotent maintenance
  script with a dry-run report (path + content duplicate groups, child
  row counts) and transactional cleanup, for servers that prefer to
  dedup manually before upgrading

Verified against the live database: the duplicate pair was collapsed
(reading_progress preserved on the survivor), schema.sql re-runs are a
no-op, and the constraint is in place with 62 unique books remaining.
2026-08-14 08:18:36 -04:00
john-okeefe b7a9b470a7 chore(deps): pin foliate-js to d4d87a9 via canonical https URL
Switch the @bookhoard/foliate-js dependency from the github: shorthand
(d164d6f) to the explicit git+https URL form (d4d87a9). The newer
revision is required by the double-page-spread support (renderer
'spread' attribute) and the explicit URL form resolves more reliably
across npm/podman builds.
2026-08-14 08:17:43 -04:00
john-okeefe f5d9578375 feat(reader): wire double-page spread setting into web reader
The double_page_spread checkbox in the reader settings panel was inert:
it had no Alpine binding, no apply logic, and no persistence. Default
was also inconsistent (false in settings-manager, absent from server
defaults).

- Add doublePageSpread state to the reader Alpine component, loaded
  from saved settings (default true)
- Add applyDoublePageSpread() which sets the renderer's 'spread'
  attribute to auto/none and persists the setting via saveSettings
- Apply the spread attribute during fixed-layout renderer init
- Bind the settings checkbox with x-model and @change
- Add double_page_spread: true to ReaderService server defaults so
  new users get the same starting value the client expects
- Also improve the PDF pan/select toolbar button: distinct smart-
  select vs pan icons, highlighted state while pan mode is active,
  and dynamic tooltips/aria-labels explaining each mode
2026-08-14 08:17:25 -04:00
john-okeefe 04e2a069d6 fix(dashboard): stop duplicating items after scan completion
Release / build-and-push (push) Successful in 5m9s
The scan-complete handler in dashboard.ts attempted to deduplicate book
cards by querying [data-media-item-id], but neither the client-side
renderBookCard nor the server-side BookCard template ever set that
attribute. As a result the dedup Set was always empty, every item from
the API response was treated as new, and all items were prepended via
insertAdjacentHTML('afterbegin', ...) on every 5-minute scan — causing
visible duplication (doubling, tripling) that only cleared on page
refresh.

Fix by replacing the fragile dedup-and-prepend logic with a per-track
full innerHTML replace. This is simpler, correctly handles items that
should be removed after a scan (the old code never removed anything),
and also removes stale sections no longer returned by the API.

Additional hardening:
- Add data-media-item-id to both renderBookCard (dashboard.ts) and the
  server-side card wrapper (dashboard.templ) so server-rendered and
  JS-rendered cards are structurally identical.
- Guard the bookhoard:scan-complete listener registration with a
  module-level boolean (scanListenerRegistered) so the handler cannot
  accumulate if Alpine ever re-inits the body subtree.
- Remove debug console.log statements from the scan handler.
2026-08-10 14:46:13 -04:00
john-okeefe 05c7431d86 feat(ui): accordion sidebar panels + wood texture preview swatches
Release / build-and-push (push) Canceled after 42s
Sidebar appearance menu improvements:

Accordion behavior:
- Lift panel open/close state to a shared 'openPanel' variable on the
  parent container so only one sidebar panel (User, Appearance, Admin,
  Sign In) can be open at a time; all can be closed.
- Admin panel still auto-opens on /admin/* pages via initial state.
- Add chevron rotation to User and Appearance panels (previously only
  Admin rotated); add a chevron to the Sign In panel for consistency.

Wood texture previews:
- Generate 48x48 WebP thumbnails (~200 bytes each) from the full-size
  PNG textures (873 KB – 1.9 MB) so the bookshelf option circles show
  the actual wood grain instead of a flat grey dot.
- Use unquoted url() in the inline style to avoid templ's double-HTML
  escaping of single quotes (SanitizeStyleAttributeValues + EscapeString
  turned url('...') into url(&#39;...) which is invalid CSS).
- 'None' keeps the flat neutral circle.

CSS resilience:
- Move the wood background-image: url() rules from the compiled
  style.css into input.css (the Tailwind source) so they survive CSS
  rebuilds instead of being silently lost.
2026-08-10 14:22:46 -04:00
john-okeefe 9a62c10803 Merge branch 'new-ui'
Release / build-and-push (push) Successful in 2m44s
# Conflicts:
#	templates/admin_library.templ
#	templates/admin_library_templ.go
#	templates/conflicts_templ.go
#	templates/unlinked_books_templ.go
2026-08-10 13:48:06 -04:00
john-okeefe 8d65e03555 fix(ui): move theme checkmark when theme changes in Appearance menu
The checkmark in the Appearance theme menu was rendered server-side
(if user.Theme == opt.Name), so it never moved after switching themes
in the browser.

- Always render the check for every theme option, hidden by default,
  using a new themeCheckClass(name, current) helper that returns the
  hidden class unless the option is the active theme.
- Give each theme button a data-theme attribute and add
  updateThemeIndicators() to web/src/theme.ts, which reads the applied
  theme from the body class (theme-<name>) and toggles the hidden class
  on each check accordingly.
- Call updateThemeIndicators() from changeTheme (before the async save
  and on failure), initializeTheme, and loadUserTheme so the menu stays
  in sync with the applied theme.
- Add unit test for themeCheckClass (templates/utils_test.go) and
  regenerate templ output.
2026-08-10 13:43:32 -04:00
john-okeefe ab465d8e0e feat(ui): adopt book-open brand icon and add SVG favicon
Bring the tighten-ui brand treatment into new-ui:

- Replace the emoji book (📚) in the sidebar header with the book-open
  icon rendered in the theme accent color, matching the tighten-ui
  header brand (templates/header.templ).
- Add web/static/favicon.svg (book-open glyph, tokyo-night accent
  #7aa2f7 stroke) and reference it from the <head> of all 27 page
  templates, so the favicon is present on login/setup/error pages too.
- Regenerate templ output for all affected templates.
2026-08-10 13:43:13 -04:00
john-okeefe 1461273162 fix(sync): wire dead token cleanup queries into daily maintenance runner
CleanupExpiredRefreshTokens and CleanupExpiredOpdsTokens were generated
by sqlc but never invoked anywhere in the codebase, so expired/revoked
tokens accumulated in the database indefinitely. The refresh-token query
was parameterized in the settings-registry work specifically so its
retention window could follow the configurable session duration, but the
periodic caller was never wired up.

annotations.go:
- Rename StartTombstonePurger to StartDailyMaintenance, which now runs
  all periodic cleanup tasks from a single 24h-tick goroutine.
- Add runDailyMaintenance helper: tombstones, then OPDS tokens, then
  refresh tokens, each logging independently so one failure never skips
  the others.
- Refresh-token retention is read from the registry (SessionDuration)
  on every tick so live admin edits are honored; guarded on the registry
  being wired so unwired test paths simply skip cleanup.
- All three queries only delete rows that are already expired or
  revoked, so active sessions are never logged out.

main.go:
- Update the call site: tombstonePurgerCancel becomes maintenanceCancel
  and calls StartDailyMaintenance.

Net footprint: still one goroutine and one ticker; the cleanup adds one
DELETE per table per day.
2026-08-10 10:43:05 -04:00
john-okeefe 598d70f735 feat(admin): editable tunable settings UI with grouped sub-sections
Replace the read-only "System Information" card (which listed hardcoded
values) with editable HTMX forms, organized so the live vs restart
distinction and related settings are visually clear.

admin_settings.templ:
- AdminSettings signature now takes liveGroups and restartGroups
  ([]SettingGroup) instead of a flat entry list.
- Remove the static System Information list. Render two cards: "Live"
  (green, applies immediately) and "Restart Required" (warning header,
  saved but only takes effect after restart).
- Within each card, TunableSettingsSection clusters entries into
  labeled sub-sections by Group (e.g. "Password Quality", "Device Rate
  Limits", "Login Lockout", "Worker Pool") with uppercase tracked
  sub-headers.
- TunableSettingRow renders an inline HTMX form per setting: a Yes/No
  select for bools, a number input with min/max for ints, text
  otherwise, posting to /admin/settings/tunable. Rows show "modified
  from default" when the value differs from the compiled default.

types.go:
- Add SettingEntry (template-local mirror of database.SettingEntry,
  keeps templates from importing database) and SettingGroup.

utils.go:
- Add GroupTunableSettings: splits a flat, group-sorted entry list into
  live and restart []SettingGroup buckets preserving source order.
  utils_test.go covers the multi-group + empty cases.

frontend.go:
- The /admin/settings page handler now loads entries from the registry,
  drops the three keys that have dedicated UI cards (default_timezone
  dropdown, scan_poll_interval_seconds, auto_scan_enabled) so they are
  not listed twice, groups the rest, and passes liveGroups/restartGroups
  into the template.
2026-08-10 08:03:02 -04:00
john-okeefe 537330e7e0 feat(app): wire settings registry into startup and admin routes
Construct the SettingsRegistry at boot, load it, and thread it through
every consumer so the configurable values take effect and stay cached.

cmd/server/main.go:
- Build the registry from the Queries handle and Load() it right after
  schema init; a load failure logs and continues (getters fall back to
  compiled defaults, so startup is never blocked).
- Wire the registry into the package-level password validator
  (SetDefaultPasswordSettings) and call SetSettings on every handler/
  service that reads tunables: AuthHandler, DeviceAuthMiddleware,
  OPDSHandler, SidecarHandler, SystemSettingsHandler,
  AnnotationService, ConversionService.
- Source the restart-time values from the registry: login lockout
  (max attempts + duration) feeds NewLoginAttemptTracker, and the new
  NewSyncQueueProcessorWithConfig / NewWorkerWithConfig take the sync
  queue and worker pool configs.

router.go:
- Config gains a Settings *database.SettingsRegistry field.
- The global auth rate limiter now reads RequestsPerMinute from
  registry.AuthRateLimit() (env stays as the enabled/disabled switch
  and as the fallback if the registry is unset).

admin_library.go:
- The HTMX scan-settings save endpoint reloads the registry after
  writing so the change is visible without a page reload.
- Add PUT /admin/settings/tunable: a small HTMX endpoint that calls
  SystemSettingsHandler.ApplySetting and returns a colored status
  snippet ("Saved" or "Saved — restart required") for the admin UI's
  per-row forms.
2026-08-10 08:02:41 -04:00
john-okeefe 885f6d8187 feat(api): unified tunable settings endpoints with typed validation
Add a single pair of admin-only endpoints that supersede the scattered
scan-settings JSON routes as the canonical way to read and write
tunable system settings. Existing legacy routes are kept working for
backward compatibility and now refresh the registry cache on write.

system_settings.go:
- GET /api/system/settings returns every known setting with full
  metadata (value, type, min, max, requires_restart, category, group,
  description, is_default) via SettingsRegistry.All().
- PUT /api/system/settings accepts {key, value}; ApplySetting() looks
  up the compiled Default for the key, runs type-aware validation
  (int range, bool parse, non-empty string, timezone via
  time.LoadLocation), upserts via UpsertSystemSetting, reloads the
  registry, and reports whether a restart is needed for the change to
  take full effect. Shared by the JSON endpoint and the HTMX endpoint.
- Legacy UpdateScanSettings / GetScanSettings / UpdateTimezoneSettings
  now reload the registry after writing and prefer the registry when
  reading, so the cache stays consistent regardless of entry point.

sidecar.go:
- SidecarHandler gains an optional registry; the timezone branch of
  UpdateSystemConfiguration (PUT /api/system/config) calls
  settings.Reload() after the write so the new value is visible
  immediately. base_url handling is unchanged.

system.go:
- Register GET/PUT /api/system/settings under the existing admin
  /api/system group.
2026-08-10 08:02:22 -04:00
john-okeefe 936a48405b refactor(background): parameterize sync queue and worker pool constructors
Split each constructor into a default-args wrapper and a config-accepting
variant so the sync queue interval/batch size and the worker pool size/
queue cap can be sourced from the settings registry at startup. These
values are constructed once at boot, so they are tagged requires_restart
in the admin UI.

queue.go:
- NewSyncQueueProcessorWithConfig(db, interval, batchSize) takes the
  flush interval and batch size as parameters; NewSyncQueueProcessor
  becomes a thin wrapper with the historical 5s / 50 defaults.

worker.go:
- NewWorkerWithConfig(numWorkers, queueCap, connManager) takes the
  queue capacity as a parameter; NewWorker becomes a thin wrapper with
  the historical cap of 100.

No behavior change for existing callers; main.go will switch to the
config-accepting variants in a follow-up wiring commit.
2026-08-10 08:02:02 -04:00
john-okeefe 757398bf15 feat(sync): make annotation tombstone TTL configurable
The 30-day retention window for soft-deleted annotations was a package
const; move it behind the registry so it can be tuned live.

annotations.go:
- AnnotationService gains an optional *database.SettingsRegistry and a
  tombstoneTTL() helper. The skip-resurrect checks and the purge cutoff
  now call it instead of reading the TombstoneTTL const directly.
- Add ActiveTombstoneTTL() so callers outside the sync package can
  compute cutoffs consistently with the service.
- The package-level TombstoneTTL const is retained as the fallback for
  tests / unwired code paths.

kobo.go, koreader.go:
- The per-book tombstone sweep cutoff now uses
  h.annotationSvc.ActiveTombstoneTTL() instead of the wsync.TombstoneTTL
  const, so both the service and the handlers honor the configured TTL.
2026-08-10 08:01:45 -04:00
john-okeefe d12911d3c8 feat(api): make device rate limits, OPDS page size, and conversion cache configurable
Move three more hardcoded values behind the settings registry. All
apply immediately on the next request (no restart needed).

device_auth.go:
- DeviceAuthMiddleware reads per-route device rate limits (sync /
  progress / metadata per minute) from the registry on each
  authenticated request via a rateLimitConfig() helper, falling back to
  the Default* constants when no registry is wired.
- The X-RateLimit-Limit response header previously hardcoded "60" for
  every request type; it now reflects the actual configured limit for
  the request type via rateLimitForRequestType().

opds.go:
- Default (50) and maximum (200) OPDS page sizes come from the
  registry's OpdsDefaultPageSize()/OpdsMaxPageSize() instead of inline
  literals, so catalog pagination can be tuned without a redeploy.

conversion_service.go:
- The 24h kepub cache lifetime is read from the registry via a
  cacheTTL() helper (was a bare 24 * time.Hour literal in the
  constructor). The field default is retained for tests that construct
  the service directly.
- conversion_service_test.go updated to assert both the field default
  and the cacheTTL() accessor return 24h.
2026-08-10 08:01:28 -04:00
john-okeefe 457a38306d feat(auth): make session duration and password rules configurable
Replace the hardcoded 7-day session lifetime and fixed password
complexity rules with registry-backed accessors so they can be tuned
from the admin UI without a code change.

auth.go:
- Drop the SessionDuration const; keep DefaultSessionDuration (7 days)
  as the fallback used when no registry is wired (e.g. in tests).
- AuthHandler gains an optional *database.SettingsRegistry and a
  sessionDuration() helper that reads the registry, falling back to
  DefaultSessionDuration.
- Cookie MaxAge, JWT exp claim, and ExpiresIn responses now derive from
  sessionDuration() instead of the package-level SessionDurationSec, so
  a settings change takes effect on the next login.

refresh_token.go:
- Refresh-token lifetime follows sessionDuration() via a new
  refreshTokenTTL() helper (was a separate refreshTokenExpiration const
  that silently had to be kept in sync with the session duration).

password_validator.go:
- PasswordValidator now reads min length and the upper/lower/number/
  special toggles from the registry at validation time, so rule
  changes apply immediately. The special-character regex is compiled
  once and reused (sync.Once).
- GetPasswordRequirements() and ValidatePassword() reflect the active
  configured rules instead of a static list.
- Add SetDefaultPasswordSettings() so the package-level default
  validator (used by echo's struct-tag validator) follows live config.

All paths degrade gracefully to the historical defaults when no
registry is wired.
2026-08-10 08:01:11 -04:00
john-okeefe bc47450653 feat(db): typed tunable system settings + SettingsRegistry
Add a typed, cached registry over the system_settings table so that
values which used to be hardcoded Go literals can be changed at runtime.

Schema (database/schema/schema.sql):
- Extend system_settings with setting_type, min_value, max_value,
  requires_restart, and category columns (all ADD COLUMN IF NOT EXISTS,
  nullable for backward compat with the original three rows).
- Seed rows for every tunable: session duration, password rules,
  login lockout, auth/device rate limits, OPDS page size, tombstone TTL,
  conversion cache TTL, sync queue interval/batch, and worker pool
  size/cap. Seed values equal the previous hardcoded literals, so
  behavior is unchanged on upgrade. ON CONFLICT DO NOTHING preserves
  any admin-modified values.

Queries (queries.sql):
- Add UpsertSystemSetting (RETURNING *) so new keys without a seed row
  can still be written through the API.
- Add GetSystemSettingFull + GetAllSystemSettingsFull returning the
  full typed row.
- Refactor CleanupExpiredRefreshTokens to take the retention window as
  a parameter (make_interval(secs => $1)) instead of the INTERVAL '7
  days' literal, so it can follow a configurable session duration.

Registry (internal/database/settings_registry.go):
- SettingsRegistry holds an in-memory cache of all known settings,
  populated by Load at startup and refreshed by Reload on writes.
- Typed domain getters (SessionDuration, PasswordRules, DeviceRateLimits,
  TombstoneTTL, OpdsPageSize, ConversionCacheTTL, SyncQueueConfig,
  WorkerPoolConfig, LoginLockout, AuthRateLimit, ...) with compiled-in
  fallback defaults and min/max clamping, so a corrupt or missing row
  can never break the app.
- SettingDefaults is the single source of truth for keys, types, bounds,
  and human descriptions; All() exposes metadata + current values for
  the admin UI/API.

The registry lives in the database package (rather than its own
internal/settings package) because a quirk in this custom go1.26.5
toolchain prevented the large handlers package from importing any
newly-created package; every consumer already imports database.

Tests: settings_registry_test.go covers default validity per type,
int clamping at both bounds, garbage-value fallback, and unknown-key
lookup.
2026-08-10 08:00:52 -04:00
john-okeefe 89ea310414 Merge remote-tracking branch 'origin/main'
# Conflicts:
#	templates/book_detail.templ
#	templates/collection_rules.templ
#	templates/conflicts.templ
#	templates/header.templ
#	templates/progress.templ
2026-08-08 15:43:16 -04:00
john-okeefe 4b152bbe4e fix(templates): use expression attributes and fix indentation
Convert string-interpolation attributes (value="{ x }") to templ
expression attributes (value={ x }) for IDs, paths, and titles, and
fix indentation in header.templ and progress.templ.
2026-08-08 15:40:15 -04:00
john-okeefe 64ad10b3f2 feat(admin): add scan settings + system information to settings page
- Scanning section: auto-scan toggle (on/off) and poll interval input
  with HTMX form that updates system_settings table
- System Information section: surfaces all hardcoded constants
  (session duration, password policy, rate limits, worker pool,
  sync queue, tombstone TTL, OPDS page size, CORS, etc.)
- New ScanSettingsData type, ScanSettingsSection partial template
- New HTMX endpoint: PUT /admin/settings/scan
- registerAdminSettingsRoutes for scan settings HTMX CRUD
- Fix bookhoord→bookhoard import typo in admin_library.go
2026-08-07 23:37:14 -04:00
john-okeefe a42232e67a fix(admin): expandable sidebar section + library manage toggle
- Admin section in sidebar now uses same expandable panel pattern as
  Appearance and User sections (toggle button with chevron, auto-opens
  when on /admin pages)
- Library Manage button toggles open/close instead of only opening
  (uses htmx.ajax for open, clears panel for close)
2026-08-07 23:29:01 -04:00
john-okeefe 006cc0c2c9 chore(frontend): slim admin.ts, remove dead library.ts
- Remove dead functions from admin.ts: loadSystemStats, renderSystemStats,
  triggerLibraryScan, triggerQuickScan, all WebSocket functions
- Remove library.ts (695 lines of innerHTML string-building replaced by
  HTMX server-rendered partials)
- Remove library import from main.ts
2026-08-07 09:35:47 -04:00
john-okeefe 706be09dec feat(admin): redesign library management with HTMX expandable rows + stats dashboard
Library management:
- Redesign admin_library page with expandable rows that load detail
  panels via HTMX (LibraryList, LibraryPanel, FolderBrowserContent partials)
- Add create/edit/delete library modals using data-* attributes
- Add folder browser modal with inline add/remove via HTMX
- Add user visibility checkboxes toggled via HTMX
- New endpoints in admin_library.go: create, update, delete, panel,
  folders add/remove, browse, visibility
- New template types: FolderData, DirEntry, UserVisibilityData,
  AdminStats; extend LibraryData with TypeValue and FolderCount

Admin dashboard:
- Rewrite admin.templ to show 4-stat grid (libraries, media, users, devices)
- Add getAdminStats helper querying library/user/media/device counts
- Pass AdminStats to template from both /admin and /admin/ handlers
2026-08-07 09:35:39 -04:00
john-okeefe a87c8afc22 refactor(admin): remove separate sidebar, integrate admin nav into main sidebar
- Delete AdminSidebar component entirely
- Add Administration section (Dashboard, Libraries, Users, Settings) to
  header sidebar, visible only for admin users
- Remove Admin Panel link from user dropdown
- Strip admin chrome (sidebar wrapper, back buttons) from all 5 admin
  page templates
- Fix activeClass to handle trailing-slash routes correctly
- Add isUserVisible helper for library visibility toggles
- Fix processing issues page: remove dead Alpine JS, wire HTMX dismiss
  with proper mediaItemId, add issue ID swap targets
- Add processing issue resolve/delete routes to library router
- Add GetProcessingIssueStatsData context-based method
- Fix users page: remove broken hx-headers auth, simplify role select
- Fix settings page: remove dead adminSettings Alpine ref
2026-08-07 09:35:30 -04:00
john-okeefe 2976dad4e5 Merge branch 'new-ui' into main
Release / build-and-push (push) Successful in 2m45s
UI/UX redesign with modern look (Kavita/Komga/Audiobookshelf vibe):
- Dashboard read-action JS fix with conflict routing
- Reader back button with sessionStorage referrer
- Collection detail page interactions (selection, bulk remove, book picker)
- Custom confirm dialog replacing native confirm()
- System collections read-only
- Collection modal icon/color selection
- OPDS base_url fix + setup gate requires base_url
- Progress page fixes (percentage formatting, blank fields)
- Devices page reform (KOReader plugin instructions, gear icon)
- Embed time/tzdata for Alpine container compatibility

# Conflicts:
#	templates/reader_templ.go
#	web/static/style.css
2026-08-06 15:24:48 -04:00
john-okeefe acc9d08c60 fix: embed time/tzdata so time.LoadLocation works in Alpine container
Alpine doesn't ship the IANA timezone database, causing
time.LoadLocation('America/New_York') to fail with 'Invalid timezone'
for every non-UTC option in the profile settings dropdown.
2026-08-06 15:18:22 -04:00
john-okeefe e09c55cecc feat(devices): reform Add Device modal with KOReader plugin instructions
- Add gear icon (proper cog) and use it for device settings button
- Replace KOReader manual identifier input with step-by-step plugin
  setup instructions (clone repo, enter server URL, approve pending reg)
- Show server URL with copy button pre-filled from baseURL
- Kobo keeps manual registration flow (device name + identifier)
- Comment out Web Browser and Mobile App options (not implemented)
- Use Alpine x-model on device-type select to toggle between
  KOReader instructions and Kobo registration form
2026-08-06 14:48:48 -04:00
john-okeefe 6f1c13998c fix(ui): populate Last Updated and Synced From fields on progress page
GetAllProgressData (SSR handler) was missing LastUpdated, DeviceIcon,
DeviceName, DeviceType, and EpubCFI fields that the template expects.
All showed blank. Now matches the API handler's field population.
2026-08-06 14:06:47 -04:00
john-okeefe 0a0caa9533 fix(ui): round progress percentage to 2 decimals on progress page
The large percentage badge displayed the raw float value unformatted
(e.g. 1.523456789%). Now uses %.2f for a clean display (e.g. 1.52%).
2026-08-06 13:25:54 -04:00
john-okeefe 4716790564 fix: OPDS base_url placeholder bug + setup gate requires base_url
Three bugs fixed:

1. Schema seeded base_url with fake placeholder 'bookhoard.example.com'.
   Removed seed; startup now seeds from BASE_URL env var only if DB row
   is empty (admin changes persist across restarts). One-time UPDATE
   clears the placeholder in existing installs.

2. config.GetBaseURL() had a broken type assertion (local SystemConfigRow
   vs database.SystemConfig) that always failed, returning . Admin panel
   showed env var fallback instead of actual DB value. Fixed with a
   function-type getter that properly wraps the DB query.

3. OPDS handler read base_url only from DB with no fallback. When DB had
   the placeholder, all feed links pointed to an unreachable domain,
   breaking KOReader search/download. Added deriveBaseURL() helper that
   falls back to the request Host/scheme when DB value is empty.

Setup gate improvements:
- isSetupComplete now requires both admin user AND non-empty base_url
- Setup middleware no longer exempts all /api/ routes; only allows
  /api/auth/register, /api/auth/login, /api/system/config before setup
  is complete. All other API routes get 503.
- Cache invalidated when base_url is saved via admin settings

Dev workflow:
- New bruno/NewDevDBSetup/SetBaseUrl.yml for dev DB setup
- NewDB.sh runs SetBaseUrl between RegisterUser and CreateEbookLibrary
2026-08-06 13:02:35 -04:00
john-okeefe 8e2c1a4b3a fix(ui): make icon and color selection work in collection modal
Three root causes, all fixed:

1. Icon buttons were created with setAttribute('onclick', ...) in
   populateIconGrid, but selectIcon is module-scoped (not on window),
   so clicking threw ReferenceError. Switch to addEventListener with
   a closure. Icon search/focus used plain oninput/onfocus attributes
   with the same problem — convert to Alpine @input/@focus.

2. selectColor's highlight selector queried [onclick="selectColor(...)\]
2026-08-06 11:35:14 -04:00
john-okeefe 4a870a7f18 fix(ui): keep search filter visible on system collections 2026-08-06 11:22:26 -04:00
john-okeefe 40dabfd788 feat(ui): hide management controls for system collections
System collections (Not Started, Continue Reading, etc.) compute
their contents dynamically from reading_progress, so manual
add/remove has no effect. Hide the search bar, Remove Selected
button, Add Books button, per-card checkboxes, and Remove buttons
when the collection is system, making the page visually read-only.

- Add IsSystem bool to CollectionData, populated from the database
  is_system_collection flag.
- Add data-is-system to #collection-data so the JS renderer can
  also conditionally omit controls on library switch.
- Wrap toolbar controls, book picker modal, card checkboxes, and
  remove buttons in if !collection.IsSystem in the template.
2026-08-06 11:08:29 -04:00
john-okeefe ef3c05714a fix(ui): custom confirm dialog, picker visibility, remove wiring
Three fixes for the collection detail page:

1. Picker modal never showed because the outer overlay div had
   style="display:none" with no x-show binding — add x-show bound
   to $store.bookPicker.isOpen plus a backdrop and click-to-close.

2. Replace native confirm() with an in-page Alpine modal for UI
   continuity. Add confirm dialog state (showConfirm, confirmMessage,
   pendingAction) and methods (requestRemoveBook, requestBulkRemove,
   executeConfirmed, closeConfirm) to the collections component.
   The actual API calls (doRemoveBook/doBulkRemove) are triggered only
   when the user confirms.

3. Rename removeBook → requestRemoveBook and bulkRemove →
   requestBulkRemove in template + JS renderer so the dialog opens
   instead of navigating or failing silently.
2026-08-06 10:55:37 -04:00
john-okeefe 816ee0ec80 fix(ui): wire up collection detail page interactions
The /collections/:id page had several broken features because three
referenced functions (removeBook, toggleBookForRemoval,
filterCollectionBooks) were never defined, and every book card was
wrapped in <a href="/media/..."> so clicking the checkbox or remove
button navigated to the book detail page instead.

Card restructure:
- Remove the <a> wrapper; title and cover are now individual links.
- Checkbox sits in a <label> with expanded click area (p-2 -m-2).
- Checkbox uses Alpine :checked/@change bound to a reactive
  selectedBooks array on the collections component.

Remove (single + bulk):
- Add removeBook(id) and bulkRemove() methods with confirm() dialogs.
- Wire the "Remove Selected" button with :disabled binding and @click.
- Selected-count badge is now Alpine-reactive (x-show/x-text).

Search within collection:
- Add filterCollectionBooks() that filters cards client-side by
  title/author via data-* attributes and @input.

Book picker ("Add Books"):
- Point the HTMX search inputs at the existing /api/media-items/search
  endpoint instead of the non-existent /api/media-items/filtered.
- Add hx-trigger="loadBooks" + hx-get to the grid so loadBooks()
  actually fires an initial request when the picker opens.
- Merge the hidden limit/offset inputs into the #book-picker-filters
  div so hx-include picks them up (was a separate <form id=filter-form>
  that nobody referenced).
- Add show_checkbox mode to handleSearchHTML: when present, render a
  new BookPickerGrid template with clickable, selectable cards instead
  of the reader BookCard.
- Fix bookPicker submit() to location.reload() instead of a non-existent
  reloadCollection HTMX event, and clearFilters() to target text inputs.
2026-08-06 10:40:18 -04:00
john-okeefe 73a2852ee3 fix(reader): back button remembers the page you came from
The reader's back button was hardcoded to the book detail page
(/media/{id}), so even when you launched the reader straight from the
dashboard the back button ignored that and sent you to the detail view.

Mirror the existing book-detail.ts referrer pattern: capture
document.referrer into sessionStorage on load (excluding other reader
pages and the reader's own URL), then override the back link's click to
navigate there. Falls back to the link's original href (/media/{id}) when
no valid referrer exists (direct URL access).
2026-08-06 09:51:04 -04:00
john-okeefe f67232a20b fix(ui): show read action on JS-rendered dashboard cards
The dashboard re-renders its sections client-side (library switch,
refresh, saving settings) via renderBookCard in dashboard.ts, which was
still the old markup with no .book-card-action overlay. So the read
button appeared on the server-rendered cards but vanished as soon as the
dashboard re-rendered, while the bookshelf (always templ-rendered) kept
working.

- Rewrite renderBookCard to match the templ BookCard: detail link plus
  the play/read action overlay, routing to the reader or the detail page
  when the book has an active conflict.
- Add has_conflict to the BookInfo TS type and stamp it in the dashboard
  sections API (GetSections) so client-rendered cards can route correctly.
- Add pointer-events-none / group-hover:pointer-events-auto to the
  client-rendered carousel nav buttons so they no longer swallow hover
  over edge cards, matching the templ fix.
2026-08-06 08:40:06 -04:00
john-okeefe 7b465ac97e fix(ui): use a grid icon for Customize Dashboard
The settings icon is a circle with radiating spokes, which reads as a
sun/light-mode toggle rather than a dashboard control. Swap it for the
grid icon, the standard dashboard-layout affordance.
2026-08-06 08:24:54 -04:00
john-okeefe 033a012069 feat(ui): group sidebar footer menus into distinct panels
The account and appearance menus sat in one container with no separation,
so expanding one made the other hard to find. Each collapsible menu now
lives in its own subtly lifted, bordered panel (.sidebar-panel) so the
expanded items stay visually contained and the menus never blend together.
Applied to the account, appearance, and sign-in menus.
2026-08-06 08:14:10 -04:00
john-okeefe 9165c4eb3f feat(ui): use a book-open icon for the card read action
Swaps the play (triangle) icon for book-open on book cards, since the
action is to start reading rather than play media.
2026-08-06 08:01:33 -04:00
john-okeefe 9ef6c5b6ed feat(ui): split book-card play action into reader/detail routing
The play button on book cards now opens the reader directly, instead of
always going to the detail page. Cards with an active progress sync
conflict route the play button to the detail page (which hosts the
conflict dialogue and resolves before writing progress), so the user is
never silently dropped into the reader with an unresolved conflict.

Backend:
- Add HasConflict to BookInfo and stamp it via ListSyncConflictsByUser
  (MarkActiveConflicts / MarkActiveConflictsSections) on the dashboard,
  bookshelf, series, tag, and search result card builders.
- Each page issues a single conflict query regardless of card count.

BookCard:
- Restructure into a detail link (cover + meta) with the play action as a
  sibling overlay using a pointer-events split: the container passes
  clicks through to detail while only the circular button routes to the
  reader. No nested anchors.
- On touch devices (hover: none) the play button stays visible.

Fix: carousel nav buttons had opacity-0 without pointer-events-none, so
they swallowed hover/clicks over book cards on the dashboard. They are
now click-through until the carousel is hovered.
2026-08-06 07:52:20 -04:00
john-okeefe 987ece38f0 fix(reader): keep footer compact on mobile and clear text under chrome
On phones the footer's progress cell rendered the full chapter title (e.g. 'Long Chapter Name · 5 / 12') in a div with no max-width or nowrap, so the text wrapped to multiple lines and ballooned the bottom bar. Combined with viewport offsets that were computed from assumed pixel heights with ~0px margin, the book text slipped underneath the bars.

Chapter label in footer: split #progress-display into two spans (progressLabel hidden on mobile via 'hidden sm:inline', progressMain always shown) and cap it with 'truncate whitespace-nowrap max-w-[5rem] sm:max-w-none' so it can never wrap or grow the bar. Phones now show just '5 / 12'; larger screens keep 'Chapter · 5 / 12'.

Viewport offset: replace the fragile hardcoded calc() with runtime measurement. Gave the chrome bars ids (reader-topbar/reader-bottombar) and added updateViewportInsets(), which sets #reader-viewport top/bottom from each bar's real offsetHeight (which already includes env(safe-area-inset-*) padding) plus a 6px margin. It runs on init and refreshes on resize, orientationchange, and via a ResizeObserver, so the content area tracks the actual chrome height on any DPI, notch, home-indicator, or zoom level instead of guessing.

Refactored formatProgress into formatProgressParts (returns {label, main}; only chapter mode sets a label) with a setProgress() helper wiring progressLabel/progressMain/progressText across the relocate, cycleProgressMode, and applyProgressMode call sites. Rebuilt reader_templ.go and style.css.
2026-08-05 21:58:42 -04:00
john-okeefe c5c2270007 fix(reader): keep footer compact on mobile and clear text under chrome
Release / build-and-push (push) Successful in 2m39s
On phones the footer's progress cell rendered the full chapter title (e.g. 'Long Chapter Name · 5 / 12') in a div with no max-width or nowrap, so the text wrapped to multiple lines and ballooned the bottom bar. Combined with viewport offsets that were computed from assumed pixel heights with ~0px margin, the book text slipped underneath the bars.

Chapter label in footer: split #progress-display into two spans (progressLabel hidden on mobile via 'hidden sm:inline', progressMain always shown) and cap it with 'truncate whitespace-nowrap max-w-[5rem] sm:max-w-none' so it can never wrap or grow the bar. Phones now show just '5 / 12'; larger screens keep 'Chapter · 5 / 12'.

Viewport offset: replace the fragile hardcoded calc() with runtime measurement. Gave the chrome bars ids (reader-topbar/reader-bottombar) and added updateViewportInsets(), which sets #reader-viewport top/bottom from each bar's real offsetHeight (which already includes env(safe-area-inset-*) padding) plus a 6px margin. It runs on init and refreshes on resize, orientationchange, and via a ResizeObserver, so the content area tracks the actual chrome height on any DPI, notch, home-indicator, or zoom level instead of guessing.

Refactored formatProgress into formatProgressParts (returns {label, main}; only chapter mode sets a label) with a setProgress() helper wiring progressLabel/progressMain/progressText across the relocate, cycleProgressMode, and applyProgressMode call sites. Rebuilt reader_templ.go and style.css.
2026-08-05 21:48:28 -04:00
john-okeefe b23f6b0bab build: regenerate stylesheet for sidebar app shell 2026-08-05 16:45:28 -04:00
john-okeefe 8ea70ea7f8 feat(ui): migrate remaining pages to the new shell
Apply the sidebar shell and bold primitives across the rest of the app so
the whole experience shares one visual language:

- Browse/organize: series (keeps stacked-covers/series-card CSS),
  collections (keeps wood-paneling + carousel classes), browse_detail.
- Account/stats: profile + form + modal, progress, analytics (stat-card
  tiles), devices, conflicts (status semantics), queue (status/priority
  badges).
- Admin: admin sidebar restyled with icons, dashboard/users/library/
  processing-issues/settings migrated to .card/.btn/.input primitives.
- Docs/setup/misc: docs (keeps prose/highlight.js), api_explorer, setup
  wizard, unlinked_books, custom_section builder.
- Modals/fragments: collection_modal, collection_rules,
  restore_system_collection_modal, filter_item, book_detail_modals — all
  overlays use --surface-overlay + --shadow-pop.

Every Alpine handler, HTMX attribute, id, name, and data-* is preserved;
only presentation changes.
2026-08-05 16:45:02 -04:00
john-okeefe 07578e0206 feat(ui): redesign dashboard, bookshelf, book detail, and auth pages
- Dashboard: page heading, bold section headers with accent icon tiles,
  carousel chevrons as SVG icons with theme-aware gradients (wood-paneling
  gradient classes preserved), and a cinematic BookCard (hover overlay with
  a quick-action button). BookCard is now fluid so it fills both the
  carousel slot and the bookshelf grid.
- Bookshelf: the filter wall becomes a search toolbar + a slide-in filter
  drawer (filtersOpen state added to the bookshelf Alpine component). Every
  filter input, the tristate cover toggle, tag autocomplete, save/load/clear
  actions, HTMX search/sort, and pagination are preserved.
- Book detail: blurred cover backdrop hero, rounded-2xl cover, bold
  typography, .btn action bar, progress/metadata cards, chip-style external
  links. All interactive rating, modals, and data-attrs preserved.
- Auth/landing: brand-gradient hero for index/login/register, icon feature
  cards, data-driven theme select (ThemeOptions). All ids (#theme-select,
  #result, #auth-result, password-requirement ids) and Alpine init preserved.
2026-08-05 16:44:52 -04:00
john-okeefe f402a3ee03 feat(ui): sidebar app shell and bold design system
Replace the top-nav with a fixed left sidebar + slim content topbar
(the Komga/Audiobookshelf layout), the signature change versus the
conservative tighten-ui branch.

- App shell: .app-sidebar (off-canvas on mobile via Alpine mobileMenuOpen,
  pinned at 16rem on lg), .app-topbar (fixed, blurred, 4rem), and
  .app-subbar for in-page sticky bars (parks under the topbar). Content is
  auto-offset via body:has(.app-sidebar) so reader.templ/error.templ (which
  have no sidebar) are untouched.
- Header rebuilt as the sidebar: logo + vertical nav (activeClass), an
  inline Appearance picker driven by ThemeOptions/WoodOptions, and an
  inline account menu / sign-in (preserving the inline htmx login). The
  search lives in the topbar so its dropdown still anchors correctly.
- Bolder primitives: .card -> rounded-2xl, cinematic .book-card-cover
  hover overlay with a quick-action affordance, .brand-gradient hero
  surface, .hero-backdrop (blurred cover) and .stat-card utilities.
2026-08-05 16:44:42 -04:00
john-okeefe f3f908eb7e feat(ui): design system foundation
Add semantic design tokens and base primitives to replace the verbose
inline var() styling that made the UI feel dated.

- Rename colliding Tailwind color tokens (bg-primary/bg-secondary) to
  semantic names (surface/surface-raised/content/content-muted/brand/line)
- Derive hover, overlay, border-strong, accent-muted and elevation tokens
  once on <body> so they adapt to every theme automatically
- Make :root mirror Tokyo Night to kill the first-paint theme flash
- Add component primitives in @layer components: .card (surface + soft
  shadow, no hard border), .btn variants, .input, .chip, .badge, .icon-btn
- Add theme-aware status/priority badges and global focus-visible styling
- Add an inline-SVG Icon() component (consistent stroke language) to
  replace the mixed emoji/SVG iconography
- Data-drive theme/wood options and add an active-nav helper
- Add skeleton shimmer + x-cloak support
2026-08-05 16:03:07 -04:00
john-okeefe 94a4facc6c chore(deps): promote golang.org/x/net to direct dependency
Release / build-and-push (push) Successful in 3m2s
Pulled in as a direct requirement by templ v0.3.1020 code generation; it was previously only an indirect dependency.
2026-08-05 15:31:48 -04:00
john-okeefe 11617c1860 fix(reader): tighten mobile header/footer and add safe-area margins
The reader chrome used the same dimensions at every screen size, and the book viewport offset was hardcoded to 52px. This made the header/footer oversized on phones and left the body text flush against (or overlapping) the bars, with no handling for notched-device safe areas.

- Add viewport-fit=cover so notched devices expose safe-area insets.
- Shrink the top bar on mobile (px-3 py-2 / text-base, scaling up at sm:) and hide the chapter title on phones (hidden sm:block sm:truncate).
- Shrink the bottom bar on mobile (tighter padding/gap, p-1.5 sm:p-2 on buttons) while keeping all controls visible.
- Replace the hardcoded top-[52px] bottom-[52px] viewport offsets with responsive calc() values (44/48px mobile, 60px at sm:) that fold in env(safe-area-inset-*), plus matching safe-area padding on the bars, so the book content always clears the chrome with a visible margin.

Regenerates reader_templ.go and rebuilds style.css.
2026-08-05 15:31:40 -04:00
john-okeefe d0040fe428 chore(deploy): add restart: unless-stopped to app container
Release / build-and-push (push) Successful in 2m26s
The app service in docker-compose.yml had no restart policy (defaults to
'no'), so if the process exited -- e.g. the watcher-leak panic fixed in
the previous commit -- the container stayed down until a manual
restart. Adding restart: unless-stopped makes the container self-recover
from crashes or host reboots, while still honoring explicit 'docker
compose down'.

Defense-in-depth alongside the scanner leak/panic fix: even if a future
unforeseen panic occurs, the app comes back automatically.
2026-07-31 10:45:53 -04:00
john-okeefe 59d5de3607 fix(scanner): eliminate fsnotify watcher leak and harden worker against panics
The bookhoard container crashed with 'panic: Failed to create file
watcher: too many open files' (media_scanner.go) after running for a few
hours, preceded by floods of 'no space left on device' from watcher.Add.

Root cause: every scan job called NewMediaScanner(), which eagerly
created an fsnotify watcher. SetFolders() then walked the entire library
tree and registered one inotify watch per directory (~3,000+ across the
libraries), and ScanFolders() registered them again during its walk. The
worker never called scanner.Close() on these ephemeral per-job scanners,
and the worker loop had no recover(), so:

  1. Leaked watchers accumulated until the kernel inotify watch cap was
     hit (ENOSPC -> 'no space left on device'), then
  2. the process fd limit (ulimit -n 1024) was exhausted, causing
     fsnotify.NewWatcher() to fail with EMFILE, and
  3. NewMediaScanner panicked on that error, taking down the whole
     process (exit code 2). With no restart policy the container stayed
     down.

The scan jobs run frequently (scan_poll_interval), so the leak built up
within hours. Note this was NOT a disk-space issue; df showed plenty free.

Fix:

- media_scanner.go: NewMediaScanner no longer creates a watcher eagerly
  (s.watcher starts nil), which removes the panic site entirely -- there
  is nothing to fail at construction. The watcher is created lazily only
  when needed.

- media_scanner.go: SetFolders gains a [?1049h(B[?7h[?25lEvery 2.0s: boolgaruda-ser8: Fri 31 Jul 2026 10:45:41 AM EDTin 0.002s (127)
sh: line 1: bool: command not found
[?12l[?25h[?1049l
[?1l> parameter. It creates
  and populates a watcher (returning an error instead of panicking) only
  when watch=true; otherwise it skips all watcher.Add calls. ScanFolders
  guards its watcher.Add with a nil check, and the WatchChanges event
  loop exits cleanly when there is no watcher (polling still runs).

- worker.go: the worker() loop now wraps each job in defer/recover() so a
  panicking job is recorded as failed and can never kill the process.

- worker.go: the three ephemeral scan handlers (processScanJob,
  processSetFoldersJob, processDirectoryScanJob) now defer scanner.Close()
  and call SetFolders(..., false), so scan jobs allocate zero watchers and
  zero inotify watches. Any pre-existing leak is also bounded by Close().

- handlers/scanner.go: the long-lived watch-mode scanners (StartScanner
  and StartWatchModeForLibrary) pass watch=true since they actually read
  watcher.Events for live change detection.

- calibre_integration_test.go: updated to the new SetFolders signature
  (watch=false, matching one-off scan usage).

Auto-add is fully preserved: new files are still detected by the periodic
poller (startBackupScan), which is independent of fsnotify and unaffected
by these changes. The watch-mode event loop remains as bonus responsiveness
when inotify is available; through Docker bind mounts where inotify is
unreliable, polling is what catches new books.
2026-07-31 10:45:41 -04:00
john-okeefe fffe0b17e6 ci(release): name Actions runs 'Release <tag>' instead of the commit message
Release / build-and-push (push) Successful in 2m34s
Add a top-level run-name so the Gitea Actions runs list shows
'Release v0.3.0' rather than the tagged commit's subject. Uses the same
expression (inputs.tag || ref_name) as the TAG env, so it resolves for
both tag pushes and manual workflow_dispatch.
2026-07-30 15:50:02 -04:00
john-okeefe 451aa48aec ci(release): auto-generate release notes via git-cliff
Release / build-and-push (push) Successful in 2m42s
Replace the image-only tag pipeline with a full release workflow that also
publishes a Gitea Release whose body is the annotated tag's message,
generated from Conventional Commits by git-cliff. No hand-written release
notes are required.

- cliff.toml: group commits (Features, Bug Fixes, Refactor, Documentation,
  Tests, Miscellaneous Tasks) with scopes and short-SHA links; emit only the
  current tag's section rather than the full history.
- .gitea/workflows/release.yml: tag-driven. Reads the release body from the
  annotated tag (git tag -l --format), so the tag message and release body are
  a single source of truth. Idempotent create/PATCH; prints the Gitea API
  error body on failure so a 403 names the missing token scope instead of
  failing silently. Adds a workflow_dispatch tag input so manual re-runs
  target the right tag instead of the default branch.
- Makefile: release VERSION=vX.Y.Z generates notes via git cliff --latest
  against a throwaway tag, then creates an annotated tag with
  --cleanup=verbatim so the markdown group headers are preserved (git's
  default cleanup strips lines starting with "#").
- release: project-attached wrapper accepting a positional version arg
  (./release 0.3.0 or ./release v0.3.0) and auto-prefixing v, for ergonomic
  one-command releases.
2026-07-30 15:40:27 -04:00
john-okeefe 5ac407057e fix(search): link results to book detail page and add cover thumbnails
Release / build-and-push (push) Successful in 2m24s
Search results navigated to /bookshelf with no filters instead of the
selected book's page. Results now link to /media/:id and display cover
thumbnails, with cover URLs resolved server-side via ResolveMediaURL.
Removes the dead selectedBook localStorage plumbing.
2026-07-30 14:55:00 -04:00
john-okeefe bf83492bf7 feat(book-detail): add Mark as Read / Unread toggle button
The book detail page had no way to mark a book finished or reset its
read state from the UI. Reading state is modelled by reading_progress
alone, where 'read' is the canonical signal percentage >= 1.0 (used by
the dashboard Recently Read collection, analytics, and sync priority).

Add a single toggle button in the action row (after Read Now) whose
label is server-rendered from completion state:
- not read  -> "Mark as Read"    -> PUT /api/media-items/:id/progress
                                       { percentage: 1.0 }
- read      -> "Mark as Unread"  -> DELETE /api/media-items/:id/progress

Mark as Unread cannot use PUT { percentage: 0 }: the progress handler
silently ignores percentage < 0.005 when existing progress > 0.01
(internal/handlers/media.go anti-regression guard), so DELETE is the
only reliable reset.

If the book has an active sync mismatch (an unresolved sync_conflicts
row), the toggle resolves it first via POST /api/conflicts/:id/resolve
before writing progress. Order matters: resolving sets resolved_at,
arming the 10-minute HasRecentConflictResolution suppression window so
the subsequent progress write does not spawn a brand-new conflict. The
resolve winner is any valid source key from the conflict data (prefers
"web"); it does not affect the final state, which the progress write
sets. A 400 "already resolved" response is tolerated.

Notes, highlights, and ratings are independent of reading_progress (they
reference media_items, not progress) and are never affected by the
toggle. After toggling the page reloads so the progress card, Sync
Progress button, and conflict banner re-render server-side.

- templates/utils.go: add conflictWinnerSource and conflictID helpers.
- templates/book_detail.templ: data-conflict-id/winner on <body> and the
  toggle button.
- web/src/book-detail.ts: toggleRead() + conflictId/conflictWinner/
  readSaving state (read from <body> in init()).
- templates/book_detail_templ.go regenerated.
2026-07-30 13:31:18 -04:00
john-okeefe 33c69e7c71 chore(templates): regenerate stale book_detail_modals templ output
Running `templ generate` to pick up the book_detail changes also
resynced book_detail_modals_templ.go, whose committed output was stale
relative to its source. The regeneration (templ v0.3.1020) reformats
boolean attribute rendering (e.g. `selected`) via
templ.ResolveAttributeValue and reflects pre-existing source additions
such as id/for label associations.

No source (.templ) change in this file; generated output only.
2026-07-30 13:08:25 -04:00
john-okeefe c2f72ca785 docs(bruno): correct rating scale and endpoint paths
The Bruno collection docs mislabeled the rating system and referenced
endpoints that do not exist.

- Update Media Rating.yml: the rating value is a 1-10 integer scale
  (displayed as 1-5 stars with half-star precision), not "typically 1-5".
- opencollection.yml: the rating routes live under
  /api/media-items/:id/rating (not /api/ratings/:media_id), GET returns
  null (not 0) when unrated, and document the PUT upsert route. Correct
  the scale to 1-10 here as well.
2026-07-30 13:08:14 -04:00
john-okeefe ca8c592496 feat(book-detail): add interactive half-star rating widget
The book detail page only displayed user ratings as static, non-clickable
stars. The full rating CRUD stack already existed in the backend
(media_ratings table, POST/GET/PUT/DELETE /api/media-items/:id/rating)
but nothing in the web UI could create or update a rating.

Replace the display-only renderStars output for the user rating with an
Alpine.js widget that:
- Renders 5 stars, each split into two transparent hit zones so the
  underlying 1-10 scale maps to half-star precision (left half = x.5,
  right half = whole star).
- Shows a live hover preview via a ratingHover state field.
- Saves the rating in place through POST /api/media-items/:id/rating
  (which upserts) and reflects the value immediately, with no full page
  reload.
- Displays the numeric value (e.g. "3.5 / 5") and a Clear button that
  issues DELETE to remove the rating.
- Reads the server-rendered value from a new data-rating attribute on
  <body> during the bookDetail component init().

The community rating block is left as a display-only renderStars render
since it is imported metadata, not a user rating.

templates/book_detail_templ.go is regenerated (also picking up templ
v0.3.1020 reformatting of the generated output).
2026-07-30 13:08:00 -04:00
john-okeefe 1f5b0d0164 docs(api): document OPDS pagination links and OpenSearch search
Update the OPDS section of the API reference to reflect the now-working
catalog:

- Document the page/per_page parameters and that paging is driven by the
  rel=next/previous/first/last links plus OpenSearch paging metadata.
- Refresh the example feed XML to show the pagination links, opensearch
  namespace/elements, and standard Atom <title>/<author> elements.
- Document the search endpoint's two modes: OpenSearch description
  (application/opensearchdescription+xml, no q) and results feed (with q),
  with an example description document.
2026-07-30 12:12:58 -04:00
john-okeefe 13cc689bff fix(opds): wire up catalog pagination links and OpenSearch search
The device catalog feed was unusable on paged OPDS clients such as
KOReader: it sliced results into pages but never advertised how to reach
the next page, so clients could only ever fetch the first page (~50 books)
and could not search the catalog.

GetDeviceCatalog:
- Emit the full set of OPDS pagination link relations (self, start, first,
  previous, next, last) pointing at catalog?page=N&per_page=M, with the
  device auth token appended for path-based auth.
- Emit OpenSearch totalResults/itemsPerPage/startIndex metadata.
- Point rel=search at the OpenSearch description (correct MIME type).

SearchDeviceCatalog now branches on the q parameter:
- No q: return an OpenSearch description document whose Url template
  contains the {searchTerms} placeholder, so clients can formulate a query.
- With q: return the existing acquisition results feed, now including
  totalResults.

A pure addCatalogPaginationLinks helper holds the page/URL logic so it can
be unit tested without a database. New handler tests cover middle/first/
last/single/empty pages (correct presence of next/previous) and token
appending.

Ordering is intentionally left unchanged (created_at DESC, grouped by
library).
2026-07-30 12:12:51 -04:00
john-okeefe 9920fd47b9 feat(opds): add OpenSearch pagination metadata and search description
Extend the OPDS feed model so clients can page through large catalogs and
discover how to search them.

Feed changes:
- Add the OpenSearch namespace (xmlns:opensearch) to all feeds.
- Add optional TotalResults/ItemsPerPage/StartIndex fields, serialized as
  <opensearch:totalResults>, <opensearch:itemsPerPage> and
  <opensearch:startIndex>, plus a SetPagination helper.
- Add OpenSearchDescription/OpenSearchUrl types and a NewSearchDescription
  constructor with GenerateXML/GenerateXMLString. This produces the
  OpenSearch description document (application/opensearchdescription+xml)
  that OPDS clients like KOReader fetch to learn the {searchTerms} search
  URL template.

These are building blocks; the handlers are wired up in a follow-up commit.

Tests cover SetPagination, omission when unset, XML emission of the
paging metadata, and OpenSearch description generation/serialization.
2026-07-30 12:12:33 -04:00
john-okeefe 26f695f480 fix(opds): correct feed tests referencing non-existent entry fields
The OPDS feed test suite did not compile or pass:

- TestNewEntry asserted on entry.Creator, but the Entry struct stores the
  creator under Author.Name (the Atom <author><name> element). Assert on
  entry.Author.Name instead.
- TestFeedGenerateXML expected <dc:title>/<dc:creator> elements, but the
  Entry struct emits standard Atom <title> and <author><name>. Update the
  expected substrings to match the actual (correct) output.

These are pre-existing assertion errors unrelated to any field being
removed; the code under test was already correct.
2026-07-30 12:12:18 -04:00
john-okeefe 05370d236a feat(ui): display media counts in library switcher
The UI had no surface showing how many media items have been imported.
Surface the total in the library switcher shown on the Dashboard, Series,
and Collections pages (via the LibrarySwitcher component) and in the
Bookshelf's inline library filter.

- Add a MediaCount field to LibraryData and a TotalMediaCount helper to
  sum counts for the "All Libraries" / "All Books" option.
- resolveLibrary() now fetches per-library counts (one query) and maps
  them onto each LibraryData entry, so the switcher reflects the active
  scope without changing the component's signature.
- Each library option renders "(N)" and the "All" option renders the
  grand total across the user's visible libraries.

The "All" total is the sum of the user's visible libraries, correctly
respecting per-user library visibility rather than a raw global count.

Regenerated templ files for library_switcher and bookshelf.
2026-07-30 11:41:13 -04:00
john-okeefe 114a4574b0 feat(db): add query to count media items per visible library
Add GetVisibleLibraryMediaCounts, which returns the media item count for
each library visible to a given user in a single GROUP BY query over
media_items. It mirrors the visibility logic in GetUserVisibleLibraries
(libraries default to visible unless an explicit false row exists) so
counts can be resolved in one round-trip instead of N per-library
lookups.

Regenerated sqlc bindings (querier.go, queries.sql.go).
2026-07-30 11:41:05 -04:00
john-okeefe 3c9e4d8126 fix(ci): use REGISTRY_TOKEN PAT secret for registry login
Gitea's auto GITHUB_TOKEN lacks the package scope needed to push to the container registry, causing the login step to fail. Switch the login password to a PAT stored as the REGISTRY_TOKEN repo Actions secret (scopes: write:package, read:package).
2026-07-29 17:07:00 -04:00
214 changed files with 24286 additions and 9800 deletions
+74 -5
View File
@@ -1,13 +1,25 @@
name: Release
# Builds and publishes the Bookhoard container image to the Gitea container registry.
# Triggered ONLY by a version tag push (pushing to main does nothing), so work-in-progress
# commits never ship. Each release publishes two image tags: the version and "latest".
# Overrides the default run name (the tagged commit's message) so the Actions
# runs list shows "Release v0.3.0" instead.
run-name: "Release ${{ gitea.event.inputs.tag || gitea.ref_name }}"
# Publishes the Bookhoard container image to the Gitea container registry AND
# creates a Gitea Release whose body is the annotated tag's message (generated
# locally by `make release VERSION=...` via git-cliff). Triggered by a version
# tag push, or manually via workflow_dispatch with a tag. Pushing to main does
# nothing, so work-in-progress commits never ship. Each release publishes two
# image tags: the version (e.g. v0.3.0) and "latest".
on:
push:
tags:
- 'v*'
workflow_dispatch:
inputs:
tag:
description: 'Tag to release (e.g. v0.3.0)'
required: true
type: string
jobs:
build-and-push:
@@ -15,9 +27,17 @@ jobs:
permissions:
contents: read
packages: write
env:
# Resolve the target tag for both triggers: explicit input on manual
# dispatch, otherwise the pushed tag ref.
TAG: ${{ gitea.event.inputs.tag || gitea.ref_name }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
# Full history ensures the tag annotation (the release notes) is present.
fetch-depth: 0
ref: ${{ gitea.event.inputs.tag || gitea.ref }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
@@ -27,7 +47,8 @@ jobs:
with:
registry: git.linuxhg.com
username: ${{ gitea.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# PAT stored as a repo Actions secret (auto GITHUB_TOKEN lacks package scope in Gitea)
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Build and push image
uses: docker/build-push-action@v5
@@ -39,5 +60,53 @@ jobs:
# Deployments default to "latest" via ${IMAGE_TAG:-latest} in docker-compose.yml;
# pin or roll back by setting IMAGE_TAG in .env.
tags: |
git.linuxhg.com/bookhoard/bookhoard:${{ gitea.ref_name }}
git.linuxhg.com/bookhoard/bookhoard:${{ env.TAG }}
git.linuxhg.com/bookhoard/bookhoard:latest
- name: Create Gitea Release
env:
# REGISTRY_TOKEN is reused for release creation because Gitea's auto
# GITHUB_TOKEN cannot create releases on this instance. The PAT must
# carry write:repository scope. Idempotent: re-runs update an existing
# release for this tag instead of failing with 409. On any HTTP error
# the API response body is printed so a 403 names the missing scope.
TOKEN: ${{ secrets.REGISTRY_TOKEN }}
REPO: ${{ gitea.repository }}
run: |
set -euo pipefail
: "${TAG:?TAG is required}"
API="https://git.linuxhg.com/api/v1/repos/${REPO}/releases"
AUTH="Authorization: token ${TOKEN}"
# Release body = the annotated tag's message (the git-cliff notes).
BODY="$(git tag -l --format='%(contents)' "${TAG}")"
# Tags containing a '-' (e.g. v0.3.0-rc1) are published as pre-releases.
PRE="false"; case "${TAG}" in *-*) PRE="true";; esac
PAYLOAD=$(jq -n \
--arg t "${TAG}" --arg n "${TAG}" --arg b "${BODY}" --argjson p "${PRE}" \
'{tag_name:$t, name:$n, body:$b, draft:false, prerelease:$p}')
# POST/PATCH the release, surfacing Gitea's error message on failure
# (e.g. "token does not have write scope") instead of failing silently.
api_call() {
local method="$1" url="$2" resp code rbody
resp="$(curl -sS -w '\n%{http_code}' -X "${method}" \
-H "${AUTH}" -H "Content-Type: application/json" \
-d "${PAYLOAD}" "${url}")"
code="$(printf '%s' "${resp}" | tail -n1)"
rbody="$(printf '%s' "${resp}" | sed '$d')"
if [ "${code}" -ge 400 ]; then
echo "::error::Release API ${code} (${method} ${url}): ${rbody}" >&2
return 1
fi
}
EXISTING_ID="$(curl -sS -H "${AUTH}" "${API}/tags/${TAG}" | jq -r '.id // empty' 2>/dev/null || true)"
if [ -n "${EXISTING_ID}" ]; then
api_call PATCH "${API}/${EXISTING_ID}"
echo "Updated existing release id=${EXISTING_ID} for ${TAG}"
else
api_call POST "${API}"
echo "Created new release for ${TAG}"
fi
+657 -228
View File
@@ -1,232 +1,661 @@
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright © 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for software and other kinds of works.
The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. We, the Free Software Foundation, use the GNU General Public License for most of our software; it applies also to any other work released this way by its authors. You can apply it to your programs, too.
When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you these rights or asking you to surrender the rights. Therefore, you have certain responsibilities if you distribute copies of the software, or if you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether gratis or for a fee, you must pass on to the recipients the same freedoms that you received. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights.
Developers that use the GNU GPL protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains that there is no warranty for this free software. For both users' and authors' sake, the GPL requires that modified versions be marked as changed, so that their problems will not be attributed erroneously to authors of previous versions.
Some devices are designed to deny users access to install or run modified versions of the software inside them, although the manufacturer can do so. This is fundamentally incompatible with the aim of protecting users' freedom to change the software. The systematic pattern of such abuse occurs in the area of products for individuals to use, which is precisely where it is most unacceptable. Therefore, we have designed this version of the GPL to prohibit the practice for those products. If such problems arise substantially in other domains, we stand ready to extend this provision to those domains in future versions of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents. States should not allow patents to restrict development and use of software on general-purpose computers, but in those that do, we wish to avoid the special danger that patents applied to a free program could make it effectively proprietary. To prevent this, the GPL assures that patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and modification follow.
TERMS AND CONDITIONS
0. Definitions.
“This License” refers to version 3 of the GNU General Public License.
“Copyright” also means copyright-like laws that apply to other kinds of works, such as semiconductor masks.
The Program” refers to any copyrightable work licensed under this License. Each licensee is addressed as “you”. “Licensees” and “recipients” may be individuals or organizations.
To “modify” a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a “modified version” of the earlier work or a work “based on” the earlier work.
A “covered work” means either the unmodified Program or a work based on the Program.
To “propagate” a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well.
To “convey” a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays “Appropriate Legal Notices” to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion.
1. Source Code.
The “source code” for a work means the preferred form of the work for making modifications to it. “Object code” means any non-source form of a work.
A “Standard Interface” means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language.
The “System Libraries” of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A “Major Component”, in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it.
The “Corresponding Source” for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work.
The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source.
The Corresponding Source for a work in source code form is that same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures.
When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified it, and giving a relevant date.
b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to “keep intact all notices”.
c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so.
A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an “aggregate” if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways:
a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b.
d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d.
A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work.
A “User Product” is either (1) a “consumer product”, which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, “normally used” refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product.
“Installation Information” for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made.
If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM).
The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying.
7. Additional Terms.
“Additional permissions” are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or authors of the material; or
e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors.
All other non-permissive additional terms are considered “further restrictions” within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11).
However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice.
Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License.
An “entity transaction” is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it.
11. Patents.
A “contributor” is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's “contributor version”.
A contributor's “essential patent claims” are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, “control” includes the right to grant patent sublicenses in a manner consistent with the requirements of this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version.
In the following three paragraphs, a “patent license” is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To “grant” such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party.
If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. “Knowingly relying” means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it.
A patent license is “discriminatory” if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU Affero General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the special requirements of the GNU Affero General Public License, section 13, concerning interaction through a network will apply to the combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of the GNU General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU General Public License “or any later version” applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU General Public License, you may choose any version ever published by the Free Software Foundation.
If the Program specifies that a proxy can decide which future versions of the GNU General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program.
Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the “copyright” line and a pointer to where the full notice is found.
bookhoard
Copyright (C) 2026 john-okeefe
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see <https://www.gnu.org/licenses/>.
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode:
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
bookhoard Copyright (C) 2026 john-okeefe
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, your program's commands might be different; for a GUI interface, you would use an “about box”.
You should also get your employer (if you work as a programmer) or school, if any, to sign a “copyright disclaimer” for the program, if necessary. For more information on this, and how to apply and follow the GNU GPL, see <https://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. But first, please read <https://www.gnu.org/philosophy/why-not-lgpl.html>.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<https://www.gnu.org/licenses/>.
+28 -1
View File
@@ -1,4 +1,4 @@
.PHONY: help test test-integration test-all rebuild rebuild-force rebuild-app rebuild-app-force rebuild-force-db clean restart up down logs ps test-env-up test-env-down verify-guidelines verify-quick
.PHONY: help test test-integration test-all rebuild rebuild-force rebuild-app rebuild-app-force rebuild-force-db clean restart up down logs ps test-env-up test-env-down verify-guidelines verify-quick release
# Include .env file for environment variables (single source of truth)
# Ignore if .env doesn't exist yet
@@ -44,6 +44,9 @@ help:
@echo "Verification:"
@echo " make verify-guidelines - Run comprehensive guidelines check"
@echo " make verify-quick - Run quick guidelines check"
@echo ""
@echo "Release:"
@echo " ./release v0.3.0 - Tag, push, and release (notes auto-generated from commits)"
# Run unit tests locally (fast, no containers)
test:
@@ -158,3 +161,27 @@ verify-guidelines:
verify-quick:
@echo "Running quick project guidelines verification..."
@./scripts/verify-quick.sh
# Create an annotated version tag carrying auto-generated release notes (git-cliff)
# and push it. The tag push triggers .gitea/workflows/release.yml, which builds the
# image and publishes a Gitea Release whose body is this tag's message. Notes come
# entirely from Conventional Commits — no hand-written message required.
#
# git-cliff's --latest needs the tag to exist to scope the notes, so we create a
# throwaway lightweight tag, generate the notes, replace it with an annotated tag,
# then push. --cleanup=verbatim keeps the markdown "###" group headers (git's
# default cleanup would strip lines starting with "#").
#
# Requires git-cliff: https://git-cliff.org/install
# Usage: make release VERSION=v0.3.0
release:
@test -n "$(VERSION)" || { echo "Usage: make release VERSION=v0.3.0"; exit 1; }
@command -v git-cliff >/dev/null 2>&1 || { echo "git-cliff not found — install: https://git-cliff.org/install"; exit 1; }
@if git rev-parse "$(VERSION)" >/dev/null 2>&1; then echo "Tag $(VERSION) already exists locally — delete it first: git tag -d $(VERSION)"; exit 1; fi
@echo "Generating release notes for $(VERSION)..."
@git tag "$(VERSION)" HEAD && \
(git cliff --latest --config cliff.toml > .release-notes.tmp && git tag -d "$(VERSION)" >/dev/null) || \
{ git tag -d "$(VERSION)" >/dev/null 2>&1; rm -f .release-notes.tmp; echo "git-cliff failed"; exit 1; }
@git tag -a --cleanup=verbatim -F .release-notes.tmp "$(VERSION)" HEAD && rm -f .release-notes.tmp
@git push origin "$(VERSION)"
@echo "Pushed $(VERSION) — Gitea Actions will build the image and publish the Release."
+17 -16
View File
@@ -1,10 +1,10 @@
# 📚 Bookhoard
# <img src="web/static/favicon.svg" width="32" alt="Bookhoard logo"> Bookhoard
A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and Tailwind CSS featuring **universal cross-device sync**, beautiful dark themes, and comprehensive media management.
## ✨ Why Bookhoard?
**🔄 Universal Sync**: Your reading progress, highlights, and notes sync automatically across all your devices - KOReader, Kobo, web, and mobile.
**🔄 Universal Sync**: Your reading position, bookmarks, highlights, and notes sync automatically between KOReader and the web - with native Kobo sync and mobile apps coming later.
**📱 Multi-Library**: Organize your ebooks, comics, and manga with per-library folders and smart collections.
@@ -52,13 +52,13 @@ The first user to register automatically becomes an admin.
### Universal Cross-Platform Sync
- **Real-Time Progress**: Turn a page on your Kindle, see it on your phone
- **Real-Time Progress**: Turn a page on your e-reader, see it in your browser
- **Format-Aware**: EPUB CFI, page numbers, percentages - all handled correctly
- **Offline Queue**: Changes sync when you reconnect, priority-processed
- **Conflict Resolution**: Smart handling when same book read on multiple devices
- **Book Matching**: Automatic matching using SHA-256, ISBN, UUID
- **OPDS Catalog**: Wireless book delivery to e-readers over Wi-Fi
- **Format Conversion**: On-the-fly EPUB→KEPUB for Kobo devices
- **Format Conversion**: On-the-fly EPUB→KEPUB conversion (for upcoming native Kobo support)
### Media Management
@@ -74,7 +74,7 @@ The first user to register automatically becomes an admin.
### Smart Collections
- **Auto-Assign Rules**: Automatically add books based on genre, author, series, tags, language, publisher, year
- **Device Shelf Mappings**: Sync collections to Kobo shelves and KOReader categories
- **Device Shelf Mappings**: Map collections to device shelves (used by native Kobo sync, coming soon)
- **Test Before Creating**: Preview which books match your rules
### Library Organization
@@ -102,8 +102,8 @@ The first user to register automatically becomes an admin.
- **[docs/user/calibre-integration.md](docs/user/calibre-integration.md)** - Calibre library integration
- **[docs/user/sync-guide.md](docs/user/sync-guide.md)** - Understanding and using universal sync
- **[docs/user/devices/kobo-setup.md](docs/user/devices/kobo-setup.md)** - Kobo e-reader configuration
- **[docs/user/devices/koreader-setup.md](docs/user/devices/koreader-setup.md)** - KOReader configuration
- **[docs/user/devices/kobo-setup.md](docs/user/devices/kobo-setup.md)** - Kobo e-reader configuration (coming soon)
- **[docs/user/user-guide.md](docs/user/user-guide.md)** - General user guide
- **[docs/user/admin-guide.md](docs/user/admin-guide.md)** - Admin features and configuration
- **[docs/user/settings-guide.md](docs/user/settings-guide.md)** - Settings and preferences
@@ -111,18 +111,19 @@ The first user to register automatically becomes an admin.
### For Developers
- **[docs/developer/api/api-reference.md](docs/developer/api/api-reference.md)** - Complete API documentation
- **[docs/contributing/DEVELOPMENT.md](docs/contributing/DEVELOPMENT.md)** - Development workflow
- **[docs/developer/android-app.md](docs/developer/android-app.md)** - Android app design & roadmap
- **[docs/contributing/development.md](docs/contributing/development.md)** - Development workflow
---
## 🎯 Supported Devices
| Platform | Sync | OPDS | Status |
| ---------------- | ---- | ---- | ------------------------ |
| **Web Browser** | ✅ | ✅ | Full support |
| **KOReader** | ✅ | ✅ | Kindle, Kobo, PocketBook |
| **Kobo Devices** | | | Clara, Libra, Sage, etc. |
| **Mobile Apps** | 🚧 | 🚧 | Coming Q2 2026 |
| Platform | Sync | OPDS | Status |
| ---------------- | ---- | ---- | ------------------------------------------------------------- |
| **Web Browser** | ✅ | ✅ | Full support |
| **KOReader** | ✅ | ✅ | Runs on Kindle, Kobo, PocketBook hardware |
| **Kobo Devices** | 🚧 | 🚧 | Native Kobo sync coming soon (use KOReader on Kobo today) |
| **Mobile Apps** | 🚧 | 🚧 | Native Android app in design ([docs](docs/developer/android-app.md)); iOS later |
---
@@ -155,20 +156,20 @@ bruno run
## 📊 Project Status
**Version**: 1.0
**License**: GPL-3.0
**License**: AGPL-3.0
**Status**: Production-ready ✅
---
## 🤝 Contributing
We welcome contributions! Please see [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) for guidelines.
We welcome contributions! Please see [docs/developer/development.md](docs/developer/development.md) for guidelines.
---
## 📄 License
GPL-3.0 - See [LICENSE](LICENSE) file for details.
AGPL-3.0 - See [LICENSE](LICENSE) file for details.
---
+1 -1
View File
@@ -1,3 +1,3 @@
#!/bin/bash
bru run --env Bookhoard --delay 500 "NewDevDBSetup/RegisterUser.yml" "NewDevDBSetup/CreateEbookLibrary.yml" "NewDevDBSetup/CreateComicLibrary.yml" "NewDevDBSetup/CreateMangaLibrary.yml" "NewDevDBSetup/AddEbookLibraryFolder.yml" "NewDevDBSetup/AddComicLibraryFolder.yml" "NewDevDBSetup/AddMangaLibraryFolder.yml" "NewDevDBSetup/ScanAllLibraries.yml"
bru run --env Bookhoard --delay 500 "NewDevDBSetup/RegisterUser.yml" "NewDevDBSetup/SetBaseUrl.yml" "NewDevDBSetup/CreateEbookLibrary.yml" "NewDevDBSetup/CreateComicLibrary.yml" "NewDevDBSetup/CreateMangaLibrary.yml" "NewDevDBSetup/AddEbookLibraryFolder.yml" "NewDevDBSetup/AddComicLibraryFolder.yml" "NewDevDBSetup/AddMangaLibraryFolder.yml" "NewDevDBSetup/ScanAllLibraries.yml"
+38
View File
@@ -0,0 +1,38 @@
info:
name: SetBaseUrl
type: http
seq: 3
http:
method: PUT
url: '{{base_url}}/api/system/config'
auth: inherit
body:
type: json
jsonBody: |-
{
"base_url": "http://localhost:8765"
}
headers:
- key: Authorization
value: Bearer {{token}}
- key: Content-Type
value: application/json
settings:
encodeUrl: true
timeout: 0
followRedirects: true
maxRedirects: 5
docs: |-
## Set Base URL
Configures the server's base_url during initial dev database setup.
Must be run after RegisterUser (which provides the auth token) and before
any library/device creation (which require setup to be complete).
**Method:** PUT
**Endpoint:** /api/system/config
**Auth:** Bearer token (from RegisterUser)
+1 -1
View File
@@ -47,7 +47,7 @@ docs: |-
- `id` (string, required): Media item UUID
**Request Body:**
- `rating` (number, required): Rating value (typically 1-5)
- `rating` (number, required): Rating value (1-10 integer scale; displayed as 1-5 stars with half-star precision)
- `review` (string, optional): Review text
**Response:** Updated rating object
+4 -3
View File
@@ -83,9 +83,10 @@ docs:
- **Update Highlight**: PUT /api/highlights/:id - Update highlight
- **Delete Highlight**: DELETE /api/highlights/:id - Remove highlight
Ratings (All Users)
- **Get Rating**: GET /api/ratings/:media_id - User's rating (returns 0 if unrated)
- **Create/Update Rating**: POST /api/ratings - Rate media item (1-5 stars, half-star precision)
- **Delete Rating**: DELETE /api/ratings/:media_id - Remove rating
- **Get Rating**: GET /api/media-items/:id/rating - User's rating (returns null if unrated)
- **Create/Update Rating**: POST /api/media-items/:id/rating - Rate media item (1-10 scale, displayed as 1-5 stars with half-star precision). POST upserts; PUT also available.
- **Update Rating**: PUT /api/media-items/:id/rating - Update rating (upsert)
- **Delete Rating**: DELETE /api/media-items/:id/rating - Remove rating
Collections (All Users)
- **List Collections**: GET /api/collections - Get user's collections
- **Get Collection**: GET /api/collections/:id - Collection details with media items
+37
View File
@@ -0,0 +1,37 @@
# git-cliff configuration — generates the body of each Gitea Release from
# Conventional Commits accumulated since the previous tag. Invoked in CI by
# orhun/git-cliff-action with --latest so only the current tag's section is
# emitted (no full history, no header — the Gitea Release title is the tag).
# Docs: https://git-cliff.org/docs/configuration
[changelog]
header = ""
body = """
{% for group, commits in commits | group_by(attribute="group") %}\
### {{ group | upper_first }}
{% for commit in commits %}\
- {% if commit.scope %}*({{ commit.scope }})* {% endif %}{{ commit.message | upper_first }} ({{ commit.id | truncate(length=7, end="") }})
{% endfor %}\
{% endfor %}\
"""
trim = true
footer = ""
[git]
conventional_commits = true
filter_unconventional = false
require_conventional = false
split_commits = false
commit_parsers = [
{ message = "^feat", group = "Features" },
{ message = "^fix", group = "Bug Fixes" },
{ message = "^perf", group = "Performance" },
{ message = "^refactor", group = "Refactor" },
{ message = "^docs", group = "Documentation" },
{ message = "^test", group = "Tests" },
{ message = "^chore|^ci", group = "Miscellaneous Tasks" },
{ message = ".*", group = "Other" },
]
filter_commits = false
tag_pattern = "v[0-9].*"
sort_commits = "oldest"
+59 -6
View File
@@ -14,6 +14,8 @@ import (
"log"
"time"
_ "time/tzdata"
"github.com/go-playground/validator/v10"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/labstack/echo/v5"
@@ -48,26 +50,72 @@ func main() {
}
log.Println("✅ Database schema initialized and verified, starting server...")
// Create login attempt tracker: 5 failed attempts = 15 minute lockout
loginAttemptTracker := ratelimit.NewLoginAttemptTracker(5, 15*time.Minute, 5*time.Minute)
// Load tunable settings from the DB into the registry. All values fall back
// to compiled defaults if a row is missing, so this never blocks startup.
registry := database.NewSettingsRegistry(queries)
if err := registry.Load(ctx); err != nil {
log.Printf("⚠️ Could not load system settings (using defaults): %v", err)
}
// Wire the registry into the package-level password validator so live
// rule changes apply to the echo struct-tag validator and ValidatePassword.
middleware.SetDefaultPasswordSettings(registry)
// Seed base_url from env var if not already configured. Uses conditional
// UPDATE so admin-set values are never overwritten on restart.
if cfg.BaseURL != "" {
_, err = dbPool.Exec(ctx, `
INSERT INTO system_config (key, value)
VALUES ('base_url', $1)
ON CONFLICT (key) DO UPDATE
SET value = EXCLUDED.value
WHERE system_config.value = ''
`, cfg.BaseURL)
if err != nil {
log.Printf("⚠️ Could not seed base_url: %v", err)
} else {
// Also seed derived URLs
for key, suffix := range map[string]string{
"opds_base_url": "/opds",
"api_base_url": "/api",
} {
_, _ = dbPool.Exec(ctx, `
INSERT INTO system_config (key, value)
VALUES ($1, $2)
ON CONFLICT (key) DO UPDATE
SET value = EXCLUDED.value
WHERE system_config.value = ''
`, key, cfg.BaseURL+suffix)
}
}
}
// Create login attempt tracker from configured (or default) lockout policy.
loginMaxAttempts, loginLockout := registry.LoginLockout()
loginAttemptTracker := ratelimit.NewLoginAttemptTracker(loginMaxAttempts, loginLockout, 5*time.Minute)
authHandler := handlers.NewAuthHandler(queries, cfg.JWTSecret, loginAttemptTracker)
authHandler.SetSettings(registry)
systemSettingsHandler := handlers.NewSystemSettingsHandler(queries)
systemSettingsHandler.SetSettings(registry)
sidecarHandler := handlers.NewSidecarHandler(queries, cfg)
sidecarHandler.SetSettings(registry)
libraryHandler := handlers.NewLibraryHandler(queries)
deviceHandler := handlers.NewDeviceHandler(queries, cfg.JWTSecret, cfg)
deviceAuthMiddleware := middleware.NewDeviceAuthMiddleware(queries)
deviceAuthMiddleware.SetSettings(registry)
processingIssuesHandler := handlers.NewProcessingIssuesHandler(queries)
hashConflictsHandler := handlers.NewHashConflictsHandler(queries)
// Create WebSocket connection manager
connManager := sync.NewConnectionManager()
progressService := sync.NewProgressService(queries, connManager)
annotationService := sync.NewAnnotationService(queries, connManager)
tombstonePurgerCancel := annotationService.StartTombstonePurger()
defer tombstonePurgerCancel()
annotationService.SetSettings(registry)
maintenanceCancel := annotationService.StartDailyMaintenance()
defer maintenanceCancel()
queueProcessor := sync.NewSyncQueueProcessor(queries)
queueProcessor := sync.NewSyncQueueProcessorWithConfig(queries, registry.SyncQueueConfig().Interval, registry.SyncQueueConfig().BatchSize)
queueProcessor.SetProgressService(progressService)
queueProcessor.SetAnnotationService(annotationService)
@@ -78,7 +126,8 @@ func main() {
libraryService.SyncAllowedExtensions(context.Background())
// Create worker for background tasks
worker := services.NewWorker(3, connManager)
workerCfg := registry.WorkerPoolConfig()
worker := services.NewWorkerWithConfig(workerCfg.Size, workerCfg.QueueCap, connManager)
services.WorkerInstance = worker
koreaderHandler := handlers.NewKOReaderHandler(queries, connManager, queueProcessor)
@@ -91,7 +140,9 @@ func main() {
queueHandler := handlers.NewQueueHandler(queries, queueProcessor)
conversionService := services.NewConversionService(queries, "/var/bookhoard/cache/kepub")
conversionService.SetSettings(registry)
opdsHandler := handlers.NewOPDSHandler(queries, libraryService, conversionService)
opdsHandler.SetSettings(registry)
collectionHandler := handlers.NewCollectionHandler(queries, libraryService, connManager)
dashboardService := services.NewDashboardService(queries)
@@ -144,6 +195,7 @@ func main() {
Echo: e,
Queries: queries,
Cfg: cfg,
Settings: registry,
DBPool: dbPool,
AuthHandler: authHandler,
LibraryHandler: libraryHandler,
@@ -151,6 +203,7 @@ func main() {
MediaHandler: mediaHandler,
MatchingHandler: matchingHandler,
ProcessingIssuesHandler: processingIssuesHandler,
HashConflictsHandler: hashConflictsHandler,
KOReaderHandler: koreaderHandler,
WSHandler: wsHandler,
ConflictHandler: conflictHandler,
+2 -2
View File
@@ -90,7 +90,7 @@ func TestCalibreLibraryScan(t *testing.T) {
// Create scanner and configure it
scanner := services.NewMediaScanner(setup.DB)
scanner.SetAdminID(adminID)
err = scanner.SetFolders([]string{tmpDir})
err = scanner.SetFolders([]string{tmpDir}, false)
require.NoError(t, err, "Failed to set scanner folders")
// Scan library
@@ -167,7 +167,7 @@ func TestCalibreLibraryScanWithoutSidecar(t *testing.T) {
// Create scanner and configure it
scanner := services.NewMediaScanner(setup.DB)
scanner.SetAdminID(adminID)
err = scanner.SetFolders([]string{tmpDir})
err = scanner.SetFolders([]string{tmpDir}, false)
require.NoError(t, err, "Failed to set scanner folders")
// Scan library
+216 -11
View File
@@ -45,11 +45,47 @@ CREATE TABLE IF NOT EXISTS system_settings (
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
-- Insert default system settings
INSERT INTO system_settings (setting_key, setting_value, description) VALUES
('scan_poll_interval_seconds', '60', 'How often to scan all libraries in minutes'),
('auto_scan_enabled', 'true', 'Whether auto-scanning is enabled system-wide'),
('default_timezone', 'UTC', 'System default timezone')
-- Extend system_settings with typed metadata so it can back the admin UI's
-- configurable tunables. All columns are nullable for backward compatibility
-- with the original three rows and any pre-existing data.
ALTER TABLE system_settings ADD COLUMN IF NOT EXISTS setting_type VARCHAR(20);
ALTER TABLE system_settings ADD COLUMN IF NOT EXISTS min_value TEXT;
ALTER TABLE system_settings ADD COLUMN IF NOT EXISTS max_value TEXT;
ALTER TABLE system_settings ADD COLUMN IF NOT EXISTS requires_restart BOOLEAN DEFAULT FALSE;
ALTER TABLE system_settings ADD COLUMN IF NOT EXISTS category VARCHAR(40);
-- Insert default system settings (original scan/timezone rows + tunables).
-- Values match the previous hardcoded literals, so behavior is unchanged on upgrade.
-- ON CONFLICT DO NOTHING preserves any admin-modified values.
INSERT INTO system_settings (setting_key, setting_value, description, setting_type, min_value, max_value, requires_restart, category) VALUES
('scan_poll_interval_seconds', '60', 'How often to scan all libraries (seconds)', 'int', '1', '3600', FALSE, 'scanner'),
('auto_scan_enabled', 'true', 'Whether auto-scanning is enabled system-wide', 'bool', NULL, NULL, FALSE, 'scanner'),
('default_timezone', 'UTC', 'System default timezone', 'string', NULL, NULL, FALSE, 'general'),
-- security / auth (live)
('session_duration_seconds', '604800', 'How long a login session stays valid', 'int', '300', '31536000', FALSE, 'security'),
('password_min_length', '8', 'Minimum password length', 'int', '1', '128', FALSE, 'security'),
('password_require_upper', 'true', 'Require at least one uppercase letter (A-Z)', 'bool', NULL, NULL, FALSE, 'security'),
('password_require_lower', 'true', 'Require at least one lowercase letter (a-z)', 'bool', NULL, NULL, FALSE, 'security'),
('password_require_number', 'true', 'Require at least one number (0-9)', 'bool', NULL, NULL, FALSE, 'security'),
('password_require_special', 'true', 'Require at least one special character', 'bool', NULL, NULL, FALSE, 'security'),
-- security / auth (restart required)
('auth_rate_limit_per_min', '10', 'Global auth API rate limit (requests per minute)', 'int', '1', '10000', TRUE, 'security'),
('login_max_attempts', '5', 'Failed login attempts before lockout', 'int', '1', '100', TRUE, 'security'),
('login_lockout_minutes', '15', 'Lockout duration after too many failed logins', 'int', '1', '10080', TRUE, 'security'),
-- api (live)
('opds_default_page_size', '50', 'Default OPDS page size', 'int', '1', '500', FALSE, 'api'),
('opds_max_page_size', '200', 'Maximum OPDS page size', 'int', '1', '1000', FALSE, 'api'),
('device_rate_sync_per_min', '60', 'Device sync requests per minute', 'int', '1', '10000', FALSE, 'api'),
('device_rate_progress_per_min', '120', 'Device progress requests per minute', 'int', '1', '10000', FALSE, 'api'),
('device_rate_metadata_per_min', '30', 'Device metadata requests per minute', 'int', '1', '10000', FALSE, 'api'),
-- sync / performance (live)
('annotation_tombstone_ttl_days', '30', 'How long deleted annotations are kept before purge', 'int', '1', '3650', FALSE, 'sync'),
('conversion_cache_ttl_hours', '24', 'How long converted (kepub) files are cached', 'int', '1', '720', FALSE, 'performance'),
-- sync / performance (restart required)
('sync_queue_interval_seconds', '5', 'How often the sync queue flushes', 'int', '1', '3600', TRUE, 'sync'),
('sync_queue_batch_size', '50', 'Maximum items processed per sync queue flush', 'int', '1', '10000', TRUE, 'sync'),
('worker_pool_size', '3', 'Number of background worker goroutines', 'int', '1', '100', TRUE, 'performance'),
('worker_queue_cap', '100', 'Background worker job queue capacity', 'int', '1', '10000', TRUE, 'performance')
ON CONFLICT (setting_key) DO NOTHING;
-- Create refresh_tokens table
@@ -1149,12 +1185,14 @@ CREATE TABLE IF NOT EXISTS system_config (
updated_by UUID REFERENCES users(id)
);
-- Pre-seeded values
INSERT INTO system_config (key, value) VALUES
('base_url', 'https://bookhoard.example.com'),
('opds_base_url', 'https://bookhoard.example.com/opds'),
('api_base_url', 'https://bookhoard.example.com/api')
ON CONFLICT (key) DO NOTHING;
-- One-time cleanup: clear the old placeholder seed so the startup logic
-- can re-seed from the BASE_URL env var (or the setup wizard can set it).
UPDATE system_config SET value = ''
WHERE key = 'base_url' AND value = 'https://bookhoard.example.com';
UPDATE system_config SET value = ''
WHERE key = 'opds_base_url' AND value = 'https://bookhoard.example.com/opds';
UPDATE system_config SET value = ''
WHERE key = 'api_base_url' AND value = 'https://bookhoard.example.com/api';
-- Create opds_tokens table (device-specific OPDS access tokens)
CREATE TABLE IF NOT EXISTS opds_tokens (
@@ -1357,3 +1395,170 @@ CREATE UNIQUE INDEX IF NOT EXISTS idx_media_bookmarks_dedup
CREATE INDEX IF NOT EXISTS idx_media_highlights_deleted_at ON media_highlights(deleted_at) WHERE deleted = TRUE;
CREATE INDEX IF NOT EXISTS idx_media_notes_deleted_at ON media_notes(deleted_at) WHERE deleted = TRUE;
CREATE INDEX IF NOT EXISTS idx_media_bookmarks_deleted_at ON media_bookmarks(deleted_at) WHERE deleted = TRUE;
-- ============================================
--: MEDIA ITEM DEDUPLICATION + PATH UNIQUENESS
-- ============================================
-- A read-then-write race in the scanner historically allowed the same
-- (library_id, file_path) to be inserted twice. This block is self-healing:
-- it collapses any existing path-duplicates (re-parenting child rows onto a
-- survivor so no reading history is lost), then enforces uniqueness going
-- forward. Idempotent — safe to re-run on every startup.
-- Move every child row that points at p_source so it points at p_target,
-- deleting source rows that would violate a UNIQUE constraint on the target.
CREATE OR REPLACE FUNCTION reparent_media_item_children(p_target UUID, p_source UUID)
RETURNS void
LANGUAGE plpgsql
AS $$
BEGIN
IF p_target IS NULL OR p_source IS NULL OR p_target = p_source THEN
RETURN;
END IF;
DELETE FROM reading_progress
WHERE media_item_id = p_source
AND user_id IN (SELECT user_id FROM reading_progress WHERE media_item_id = p_target);
UPDATE reading_progress SET media_item_id = p_target WHERE media_item_id = p_source;
DELETE FROM reading_speed
WHERE media_item_id = p_source
AND user_id IN (SELECT user_id FROM reading_speed WHERE media_item_id = p_target);
UPDATE reading_speed SET media_item_id = p_target WHERE media_item_id = p_source;
DELETE FROM media_ratings
WHERE media_item_id = p_source
AND user_id IN (SELECT user_id FROM media_ratings WHERE media_item_id = p_target);
UPDATE media_ratings SET media_item_id = p_target WHERE media_item_id = p_source;
DELETE FROM media_bookmarks
WHERE media_item_id = p_source
AND (user_id, title) IN (SELECT user_id, title FROM media_bookmarks WHERE media_item_id = p_target);
UPDATE media_bookmarks SET media_item_id = p_target WHERE media_item_id = p_source;
DELETE FROM media_item_formats
WHERE media_item_id = p_source
AND format_type IN (SELECT format_type FROM media_item_formats WHERE media_item_id = p_target);
UPDATE media_item_formats SET media_item_id = p_target WHERE media_item_id = p_source;
DELETE FROM collection_items
WHERE media_item_id = p_source
AND collection_id IN (SELECT collection_id FROM collection_items WHERE media_item_id = p_target);
UPDATE collection_items SET media_item_id = p_target WHERE media_item_id = p_source;
DELETE FROM kobo_shelves
WHERE media_item_id = p_source
AND device_id IN (SELECT device_id FROM kobo_shelves WHERE media_item_id = p_target);
UPDATE kobo_shelves SET media_item_id = p_target WHERE media_item_id = p_source;
DELETE FROM panel_data
WHERE media_item_id = p_source
AND page_number IN (SELECT page_number FROM panel_data WHERE media_item_id = p_target);
UPDATE panel_data SET media_item_id = p_target WHERE media_item_id = p_source;
DELETE FROM processing_issues
WHERE media_item_id = p_source
AND issue_type IN (SELECT issue_type FROM processing_issues WHERE media_item_id = p_target);
UPDATE processing_issues SET media_item_id = p_target WHERE media_item_id = p_source;
DELETE FROM device_file_aliases
WHERE media_item_id = p_source
AND (device_id, file_path) IN (SELECT device_id, file_path FROM device_file_aliases WHERE media_item_id = p_target);
UPDATE device_file_aliases SET media_item_id = p_target WHERE media_item_id = p_source;
-- Tables whose UNIQUE keys do not include media_item_id.
UPDATE device_catalogs SET media_item_id = p_target WHERE media_item_id = p_source;
UPDATE kobo_entitlements SET media_item_id = p_target WHERE media_item_id = p_source;
UPDATE media_highlights SET media_item_id = p_target WHERE media_item_id = p_source;
UPDATE media_notes SET media_item_id = p_target WHERE media_item_id = p_source;
UPDATE reading_history SET media_item_id = p_target WHERE media_item_id = p_source;
UPDATE sync_conflicts SET media_item_id = p_target WHERE media_item_id = p_source;
UPDATE sync_queue SET media_item_id = p_target WHERE media_item_id = p_source;
END;
$$;
-- Collapse every (library_id, file_path) group into a single row.
-- Survivor = the row with the most user data; ties broken by lowest id.
CREATE OR REPLACE FUNCTION dedup_media_items_by_path() RETURNS void
LANGUAGE plpgsql
AS $$
DECLARE
g RECORD;
v_surv UUID;
v_loser UUID;
BEGIN
FOR g IN
SELECT library_id, file_path
FROM media_items
GROUP BY library_id, file_path
HAVING COUNT(*) > 1
LOOP
SELECT mi.id INTO v_surv
FROM media_items mi
WHERE mi.library_id = g.library_id AND mi.file_path = g.file_path
ORDER BY
((SELECT COUNT(*) FROM reading_progress rp WHERE rp.media_item_id = mi.id)
+ (SELECT COUNT(*) FROM media_highlights mh WHERE mh.media_item_id = mi.id)
+ (SELECT COUNT(*) FROM media_bookmarks mb WHERE mb.media_item_id = mi.id)
+ (SELECT COUNT(*) FROM media_notes mn WHERE mn.media_item_id = mi.id)
+ (SELECT COUNT(*) FROM reading_history rh WHERE rh.media_item_id = mi.id)
+ (SELECT COUNT(*) FROM collection_items ci WHERE ci.media_item_id = mi.id)) DESC,
mi.id ASC
LIMIT 1;
FOR v_loser IN
SELECT id FROM media_items
WHERE library_id = g.library_id AND file_path = g.file_path AND id <> v_surv
ORDER BY id
LOOP
PERFORM reparent_media_item_children(v_surv, v_loser);
DELETE FROM media_items WHERE id = v_loser;
END LOOP;
END LOOP;
END;
$$;
-- Collapse any existing path-duplicates so the constraint below can be created.
SELECT dedup_media_items_by_path();
-- Enforce path uniqueness going forward (guarded so re-runs don't error).
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_constraint
WHERE conname = 'media_items_library_id_file_path_key'
AND conrelid = 'media_items'::regclass
) THEN
ALTER TABLE media_items
ADD CONSTRAINT media_items_library_id_file_path_key UNIQUE (library_id, file_path);
END IF;
END $$;
-- ============================================
--: HASH CONFLICTS
-- ============================================
-- Records content-duplicate groups discovered during hash backfill or rescan:
-- two or more media_items in the same library share a file_sha256 but live at
-- different file paths (e.g. the same book imported twice under two names on
-- a preexisting database). Unlike path duplicates these cannot be auto-collapsed
-- (keeping both copies may be intentional), so each group is surfaced on the
-- admin Hash Conflicts page for the user to resolve:
-- keep_all - both copies are intentional; just stop flagging
-- kept:<uuid> - merge every other copy's child rows into the kept item
-- (via reparent_media_item_children) and delete the losers
CREATE TABLE IF NOT EXISTS hash_conflicts (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
library_id UUID NOT NULL REFERENCES libraries(id) ON DELETE CASCADE,
file_sha256 CHAR(64) NOT NULL,
status VARCHAR(20) NOT NULL DEFAULT 'pending' CHECK (status IN ('pending','resolved')),
resolution VARCHAR(50), -- 'keep_all' or 'kept:<media_item_uuid>' (41 chars)
resolved_by UUID REFERENCES users(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ DEFAULT NOW(),
resolved_at TIMESTAMPTZ,
UNIQUE(library_id, file_sha256)
);
CREATE INDEX IF NOT EXISTS idx_hash_conflicts_status ON hash_conflicts(status);
-- Widen for databases created before the resolution format settled (no-op otherwise)
ALTER TABLE hash_conflicts ALTER COLUMN resolution TYPE VARCHAR(50);
+1
View File
@@ -31,6 +31,7 @@ services:
app:
image: git.linuxhg.com/bookhoard/bookhoard:${IMAGE_TAG:-latest}
container_name: bookhoard
restart: unless-stopped
environment:
# Database Configuration
DATABASE_HOST: db
+153
View File
@@ -0,0 +1,153 @@
# Android App — Design & Roadmap
This document describes the planned native Android client for Bookhoard: a thin, offline-first reading app that treats the Bookhoard server as its backend. The relationship is the same as the audiobookshelf app to an audiobookshelf server, or the Kindle app to Kindle cloud — the server owns the library, sync, and conflict resolution; the app is a dedicated, mobile-first reading frontend with its own UI, designed independently of the web interface.
**Status**: Planning / pre-development
**Companion repo**: `bookhoard-app` (separate repository, AGPL-3.0)
---
## 🎯 Product Vision
- An **amazing ereader** first and foremost — rendering polish, latency, and reading UX are the product
- **Mobile-first UI** designed from scratch for phones; not a wrapper around the web app
- **Thin client**: the server remains authoritative for all sync, book matching, and conflict resolution
- **v1 formats**: EPUB (ebooks) and CBZ (comics/manga); PDF comes nearly free via the reader toolkit
- **Android first**. iOS is a real roadmap item but unscheduled — likely contributor-driven
---
## 🛠 Tech Stack
| Concern | Choice |
| -------------- | ------------------------------------------------- |
| Language | Kotlin |
| UI | Jetpack Compose + Material 3 |
| Reader engine | [Readium Kotlin toolkit](https://github.com/readium/kotlin-toolkit) |
| Local database | Room |
| Networking | OkHttp / Retrofit + WebSocket |
| Background | WorkManager |
| Images | Coil |
| Settings | DataStore |
### Why native Android
- The quality bar is the Kindle app. Page-turn latency, text layout fidelity, PDF rendering (`PdfRenderer`), and comic/manga image pipelines are platform-level strengths — and they are the *hard* parts in a WebView, not the easy parts.
- Android-first removes the "share one codebase across two platforms simultaneously" constraint that motivates hybrid stacks.
- Solo, AI-assisted development compresses the cost of native (code volume), while native's failure modes (well-documented platform APIs) are far easier to debug — alone or with AI — than cross-framework bridge/plugin bugs.
- The target audience is the self-hosted community, best reached via GitHub Releases and F-Droid rather than app-store optimization.
### Alternatives considered
- **Capacitor / WebView shell** (the audiobookshelf-app model): excellent when a self-contained SPA already exists; a poor fit here. Bookhoard's web UI is server-rendered HTMX and cannot be packaged, comics rendering in a WebView caps the polish target, and deep offline support fights the shell.
- **Flutter**: strong middle ground, but no Readium port and a weaker EPUB/PDF plugin ecosystem than the native toolkits.
- **Kotlin Multiplatform**: only pays off with a committed near-term iOS effort. Revisit if iOS becomes active; until then it would constrain v1 for a hypothetical.
---
## 🏗 Architecture
Thin, offline-first client. The server API is the contract (see [API Reference](api/api-reference.md) and [WebSocket API](websocket-api.md)).
### Module layout
```
:app Compose UI, navigation, dependency injection
:core:domain Pure Kotlin — models, sync logic, use cases (no Android deps)
:core:data Room, Retrofit/OkHttp, downloads and file storage
:feature:reader Readium navigator integration and reading UI
```
Keeping `:core:domain` free of Android dependencies preserves optionality: a future iOS client, a KMP extraction, or a desktop client can reuse or port the domain logic without touching the UI.
### Offline-first sync flow
1. UI writes go to the local Room mirror **first** (never blocked on network)
2. A WorkManager queue replays changes to the existing REST endpoints (`/api/progress`, `/api/media-items/:id/notes`, `/highlights`, etc.)
3. Conflicts are resolved by the server's existing mechanisms — the client never invents its own merge logic
4. While online, a WebSocket connection receives realtime updates pushed by other devices (web reader, KOReader)
5. Books are downloaded to app storage for fully offline reading, with storage management UI
### Authentication & device identity
- **Primary auth: username/password login** via the existing endpoints (`POST /api/auth/login` + refresh). The app is a full user client — browse, collections, ratings, and annotation management all live behind the user JWT, which device tokens cannot reach
- After login, the app registers itself as a **device** (`device_type: mobile`) and **self-approves** its registration using its own JWT — approval only requires a logged-in user. The phone then appears on the Devices page with sync attribution, per-device settings, and individually revocable access, with no QR ceremony
- Netflix-style QR pairing as a zero-typing sign-in option: post-v1 (see below)
---
## 📖 Reader Engine
[Readium](https://github.com/readium/kotlin-toolkit) provides EPUB, PDF, and CBZ through one publication model and navigator — production-hardened by real reading apps. This avoids building and maintaining three renderers.
Planned reading features:
- Custom fonts (including user-loaded), adjustable margins and line height
- Themes including OLED true-black for battery
- Paginated and scroll modes; gesture and volume-key page turns
- Keep-screen-awake while reading
- Highlights, notes, and bookmarks synced via existing APIs (including deleted-annotation restore)
- Resume to exact position using EPUB CFI, consistent with universal sync
### Comics & manga UX
- RTL reading direction and double-page spreads with correct cover/single-page handling
- Per-book reading-mode overrides (a manga library can default to RTL)
- Zoom and pan; aggressive preloading of adjacent pages
- Webtoon / continuous vertical mode: post-v1
---
## 🍎 iOS Posture
iOS is a real roadmap item but not near-term. The strategy is **not** to pre-pay for it with KMP or a cross-platform framework. Instead:
- The documented REST/WebSocket API is the sharing mechanism — a future iOS client is a *new client over the same contract*, never a rewrite of shared logic
- A contributor-driven Swift/SwiftUI client is welcome; the server needs no changes to support it
---
## 📦 Distribution & Licensing
- **License**: AGPL-3.0, matching the Bookhoard server
- **Channels**: GitHub Releases and F-Droid; Play Store optional later
---
## 🚧 Milestones
1. **Scaffold** — app shell, auth + QR device pairing, library browsing, book downloads
2. **EPUB reading** — Readium integration, CFI progress sync, offline-first reading
3. **Annotations** — highlights/notes/bookmarks sync with offline queue
4. **Comics** — CBZ navigator with manga modes (RTL, spreads, zoom)
5. **Polish** — OLED themes, gestures, background sync, storage management
---
## 🔭 Post-v1 Ideas
### QR pairing sign-in (Netflix-style)
"Add device" on the web (while logged in) displays a QR code; a fresh app install scans it and is **fully signed in** — no server URL, no password, nothing typed on the phone.
- **QR is a full login**: the claim endpoint returns JWT + refresh token (plus the device token for sync identity)
- **Typed-code fallback** (GitHub/Netflix device-flow style: app displays a short code, user enters it on the web) for phones with broken cameras or no camera
- **KOReader keeps its existing flow unchanged** — no typed-code pairing there; it is already as convenient as it can be
- **Use the configured `BASE_URL`, never a detected LAN IP** — if the server is published at `https://public.domain`, pairing must work identically from outside the LAN
- Requires small server additions: `pair`/`claim` endpoints backed by single-use pairing sessions with a short TTL (in-memory like `pendingRegistrations`)
### Other ideas
- Webtoon / continuous vertical reading mode
- Home-screen widgets and app shortcuts ("continue reading")
- Text-to-speech
- OPDS feed consumption from other servers
---
## Related Documentation
- **[API Reference](api/api-reference.md)** - Complete REST API
- **[WebSocket API](websocket-api.md)** - Real-time sync events
- **[Sync Guide](../user/sync-guide.md)** - How universal sync works
- **[Devices API](api/devices/)** - Device registration and approval
+173 -24
View File
@@ -26,14 +26,16 @@ Complete API documentation for Bookhoard v1.0 with Universal Cross-Platform Sync
8. [Device Management](#device-management)
9. [Analytics](#analytics)
10. [Book Matching & Linking](#book-matching--linking)
11. [Collections](#collections) → See [COLLECTIONS_API.md](COLLECTIONS_API.md)
11. [Collections](#collections) → See [Collections API](collections-api.md)
12. [OPDS](#opds-open-publication-distribution-system)
13. [Sync Protocol - KOReader](#sync-protocol---koreader)
14. [Sync Protocol - Kobo](#sync-protocol---kobo)
15. [Universal Progress](#universal-progress)
16. [Conflicts](#conflicts)
17. [Sync Queue](#sync-queue)
18. [WebSocket](#websocket)
18. [System Settings & Configuration](#system-settings--configuration)
19. [Hash Conflicts](#hash-conflicts)
20. [WebSocket](#websocket)
## Base URL
@@ -201,7 +203,9 @@ Content-Type: application/json
}
```
### Update Scan Settings
### Update Scan Settings (Legacy)
> Superseded by `PUT /api/system/settings` (see [System Settings & Configuration](#system-settings--configuration)); kept for backward compatibility.
```http
PUT /api/libraries/scan-settings
@@ -665,18 +669,23 @@ Content-Type: application/json
```json
{
"device_id": "uuid",
"registration_id": "registration-uuid",
"auth_url": "https://bookhoard.com/devices/auth/confirm/abc123",
"auth_url": "https://bookhoard.com/devices/approve/abc123",
"qr_code": "data:image/png;base64,iVBORw0KG...",
"expires_in": 300
"expires_in": 300,
"poll_interval": 3,
"setup_instructions": {
"koreader": "Calibre URL: https://bookhoard.com/api/sync/koreader"
}
}
```
Open `auth_url` (or scan the QR code) while logged in to approve; the registration expires after 5 minutes.
### Check Registration Status
```http
POST /api/devices/auth/status
POST /api/devices/register/status
Content-Type: application/json
{
@@ -688,13 +697,13 @@ Content-Type: application/json
```json
{
"status": "pending|approved|expired",
"status": "pending|approved",
"auth_token": "device-bearer-token...",
"device_id": "uuid",
"sync_endpoints": {
"progress": "https://bookhoard.com/api/sync/progress",
"metadata": "https://bookhoard.com/api/sync/metadata",
"annotations": "https://bookhoard.com/api/sync/annotations"
"progress": "https://bookhoard.com/api/sync/koreader/progress",
"metadata": "https://bookhoard.com/api/sync/koreader/metadata",
"bookmarks": "https://bookhoard.com/api/sync/koreader/bookmarks"
}
}
```
@@ -747,6 +756,22 @@ DELETE /api/devices/{device_id}
Authorization: Bearer <token>
```
### Get Device Sidecar Config
Returns the `.bookhoard.json` sidecar config for a device (server endpoints, books keyed by per-format SHA-256, collections) used by the KOReader plugin to self-configure.
```http
GET /api/devices/{device_id}/sidecar
Authorization: Bearer <token>
```
Also available as a file download:
```http
GET /api/devices/{device_id}/sidecar/download
Authorization: Bearer <token>
```
## Analytics
### Get Reading Statistics
@@ -953,7 +978,7 @@ Authorization: Bearer <token>
## Collections
For complete collection management documentation, see **[COLLECTIONS_API.md](COLLECTIONS_API.md)**.
For complete collection management documentation, see **[Collections API](collections-api.md)**.
**Quick Reference**:
@@ -986,25 +1011,39 @@ GET /opds/devices/{deviceId}/catalog?page={page}&per_page={per_page}
- `page` (optional): Page number (default: 1)
- `per_page` (optional): Items per page (default: 50, max: 200)
The feed is paginated via standard OPDS link relations. Clients (e.g. KOReader)
walk pages by following the `rel="next"` link until it is absent. OpenSearch
paging metadata (`totalResults`, `itemsPerPage`, `startIndex`) is also included.
**Response** (200 - OPDS 1.2 XML):
```xml
<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:opds="http://opds-spec.org/2010/"
xmlns:dc="http://purl.org/dc/elements/1.1/">
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">
<id>urn:uuid:device-id</id>
<title>Bookhoard Library</title>
<updated>2026-02-01T12:00:00Z</updated>
<link rel="self" href="http://localhost:8765/opds/devices/kobo-id/catalog"/>
<link rel="search" href="http://localhost:8765/opds/devices/kobo-id/search"/>
<link rel="start" href="http://localhost:8765/opds/devices/kobo-id/nav"/>
<link rel="self" href="http://localhost:8765/opds/devices/kobo-id/catalog?page=2&per_page=50"/>
<link rel="start" href="http://localhost:8765/opds/devices/kobo-id/catalog?page=1&per_page=50"/>
<link rel="first" href="http://localhost:8765/opds/devices/kobo-id/catalog?page=1&per_page=50"/>
<link rel="previous" href="http://localhost:8765/opds/devices/kobo-id/catalog?page=1&per_page=50"/>
<link rel="next" href="http://localhost:8765/opds/devices/kobo-id/catalog?page=3&per_page=50"/>
<link rel="last" href="http://localhost:8765/opds/devices/kobo-id/catalog?page=37&per_page=50"/>
<link rel="search" type="application/opensearchdescription+xml"
href="http://localhost:8765/opds/devices/kobo-id/search"/>
<opensearch:totalResults>1814</opensearch:totalResults>
<opensearch:itemsPerPage>50</opensearch:itemsPerPage>
<opensearch:startIndex>51</opensearch:startIndex>
<entry>
<id>urn:uuid:bookhoard-uuid-123</id>
<dc:title>The Hobbit</dc:title>
<dc:creator>J.R.R. Tolkien</dc:creator>
<title>The Hobbit</title>
<author><name>J.R.R. Tolkien</name></author>
<updated>2026-02-01T10:00:00Z</updated>
<link href="http://localhost:8765/opds/devices/kobo-id/download/uuid-123"
@@ -1043,10 +1082,28 @@ GET /opds/devices/{deviceId}/download/{bookId}?format={format}
### Search OPDS Catalog
```http
GET /opds/devices/{deviceId}/search?q={query}
GET /opds/devices/{deviceId}/search # OpenSearch description
GET /opds/devices/{deviceId}/search?q={query} # search results feed
```
**Response** (200 - OPDS 1.2 XML with search results)
When called **without** a `q` parameter, returns an OpenSearch description
document (`application/opensearchdescription+xml`). OPDS clients fetch this to
learn the search URL template, then substitute `{searchTerms}`:
```xml
<?xml version="1.0" encoding="UTF-8"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
<ShortName>Bookhoard</ShortName>
<Description>Search the Bookhoard library</Description>
<InputEncoding>UTF-8</InputEncoding>
<OutputEncoding>UTF-8</OutputEncoding>
<Url type="application/atom+xml;profile=opds-catalog;kind=acquisition"
template="http://localhost:8765/opds/devices/kobo-id/search?q={searchTerms}"/>
</OpenSearchDescription>
```
When called **with** a `q` parameter, **Response** (200 - OPDS 1.2 XML with
search results, including `opensearch:totalResults`).
### List Available Formats
@@ -1171,6 +1228,8 @@ Authorization: Bearer <device_token>
## Sync Protocol - Kobo
> **Status: Coming Soon** — Native Kobo sync is implemented server-side but not yet supported on real devices. These endpoints are under active development and may change.
### Kobo Markup Sync
```http
@@ -1506,6 +1565,96 @@ Authorization: Bearer <token>
}
```
## System Settings & Configuration
### List All Settings
Returns every tunable setting with current value and metadata (type, range, category, group, description, `requires_restart`, `is_default`).
```http
GET /api/system/settings
Authorization: Bearer <admin_token>
```
**Response** (200):
```json
[
{
"key": "scan_poll_interval_seconds",
"value": "60",
"type": "int",
"min": "1",
"max": "3600",
"requires_restart": false,
"category": "scanner",
"group": "Scanning",
"description": "How often to scan all libraries (seconds)",
"is_default": true
}
]
```
### Update a Setting
Type-aware validation (int range, bool parse, IANA timezone for `default_timezone`), persists the value, reloads the registry, and reports whether a restart is needed.
```http
PUT /api/system/settings
Authorization: Bearer <admin_token>
Content-Type: application/json
{
"key": "scan_poll_interval_seconds",
"value": "30"
}
```
**Response** (200): the updated entry plus `reload_required`.
Setting categories: scanner (`scan_poll_interval_seconds`, `auto_scan_enabled`), general (`default_timezone`), security (session duration, password rules, auth rate limit, login lockout), api (OPDS page sizes, device rate limits), sync (annotation tombstone TTL, sync queue interval/batch), performance (conversion cache TTL, worker pool size/capacity). See [System Settings API](api/system/settings.md) for the full catalog.
### Get / Update Raw System Config
Flat key/value configuration (e.g. `base_url`), including keys without registry metadata.
```http
GET /api/system/config
PUT /api/system/config
Authorization: Bearer <admin_token>
```
## Hash Conflicts
Duplicate content discovered during hashing (import, rescan, or the startup backfill) is grouped into hash conflicts for an explicit keep/merge decision. Files on disk are never deleted.
### List Hash Conflicts
```http
GET /api/admin/hash-conflicts
Authorization: Bearer <admin_token>
```
**Response** (200): `{ "conflicts": [ { id, library_id, library_name, sha256, created_at, items: [ { id, title, author, file_path, file_size, created_at, progress_count, highlight_count, bookmark_count, note_count, collection_count } ] } ], "total": n }`
### Resolve Hash Conflict
```http
POST /api/admin/hash-conflicts/:id/resolve
Authorization: Bearer <admin_token>
Content-Type: application/json
{
"action": "keep",
"keep_uuid": "media-item-uuid-to-keep"
}
```
- `action=keep` — merge every other copy's child rows (progress, highlights, bookmarks, notes, collections) into the kept item, then delete the losers
- `action=keep_all` — copies are intentional; dismiss the conflict
**Errors**: `400` (bad ID / missing `keep_uuid`), `404` (not found), `409` (already resolved).
## WebSocket
### Connect to WebSocket
@@ -1661,10 +1810,10 @@ bruno run bruno/devices/
## Additional Resources
- [README.md](README.md) - Getting started guide
- [UNIVERSAL_SYNC_IMPLEMENTATION_GUIDE.md](UNIVERSAL_SYNC_IMPLEMENTATION_GUIDE.md) - Sync architecture
- [KOBOREADER_SETUP.md](KOBOREADER_SETUP.md) - KOReader device setup
- [KOBO_SETUP.md](KOBO_SETUP.md) - Kobo device setup
- [README.md](../../README.md) - Getting started guide
- [Sync Guide](../user/sync-guide.md) - Sync concepts and conflict resolution
- [KOReader Setup](../user/devices/koreader-setup.md) - KOReader device setup
- [Kobo Setup](../user/devices/kobo-setup.md) - Kobo device setup (native sync coming soon)
---
+111
View File
@@ -0,0 +1,111 @@
# Hash Conflicts API
## Overview
When Bookhoard hashes your library (on import, rescan, or the startup backfill), two media items in the same library with the same `file_sha256` indicate duplicate content. Each duplicate group is recorded as a **hash conflict** and exposed here for an explicit keep/merge decision. Conflicts are also surfaced in the admin UI's Hash Conflicts page.
**Authentication**: Admin JWT token required
**Content-Type**: `application/json` (resolve also accepts form-encoded bodies for htmx)
---
## Endpoints
### List Hash Conflicts
List all pending conflict groups, each with its member items and per-item usage counts (reading progress, highlights, bookmarks, notes, collections) to help decide which copy to keep.
**Endpoint**: `GET /api/admin/hash-conflicts`
**Response**: **200 OK**
```json
{
"conflicts": [
{
"id": "conflict-uuid",
"library_id": "library-uuid",
"library_name": "Ebooks",
"sha256": "abc123...",
"created_at": "2026-08-14T12:00:00Z",
"items": [
{
"id": "media-item-uuid",
"title": "The Hobbit",
"author": "J. R. R. Tolkien",
"file_path": "/books/hobbit.epub",
"file_size": 1048576,
"created_at": "2026-01-01T00:00:00Z",
"progress_count": 2,
"highlight_count": 12,
"bookmark_count": 3,
"note_count": 1,
"collection_count": 2
}
]
}
],
"total": 1
}
```
**Example**:
```bash
curl -X GET https://bookhoard.example.com/api/admin/hash-conflicts \
-H "Authorization: Bearer <admin_token>"
```
---
### Resolve Hash Conflict
Resolve one conflict group.
**Endpoint**: `POST /api/admin/hash-conflicts/{id}/resolve`
**Request Body** (JSON or form-encoded):
```json
{
"action": "keep",
"keep_uuid": "media-item-uuid-to-keep"
}
```
| Field | Type | Required | Description |
| ----------- | ------ | -------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| `action` | string | Yes | `keep_all` — both copies are intentional; dismiss the conflict. `keep` — keep `keep_uuid` and delete the other copies. |
| `keep_uuid` | string | for `action=keep` | The media item UUID to keep. Must belong to this conflict group. With `keep`, every other copy's child rows (progress, highlights, bookmarks, notes, collections, …) are merged into the kept item before the losers are deleted. |
**Responses**:
- `200 OK` — resolved (body is an HTML confirmation snippet for the admin UI page)
- `400 Bad Request` — invalid conflict ID, missing `keep_uuid`, or `keep_uuid` not in the group
- `404 Not Found` — conflict doesn't exist
- `409 Conflict` — conflict already resolved
**Example**:
```bash
curl -X POST https://bookhoard.example.com/api/admin/hash-conflicts/<id>/resolve \
-H "Authorization: Bearer <admin_token>" \
-H "Content-Type: application/json" \
-d '{"action": "keep", "keep_uuid": "media-item-uuid"}'
```
---
## When Conflicts Are Created
- **Startup backfill**: items imported before hashing existed are hashed automatically ~30s after startup; duplicates discovered land here.
- **Rescan**: hashes are recomputed and content duplicates are flagged.
Files on disk are never deleted — resolution only affects database rows.
---
## Related Endpoints
- [System Settings API](../system/settings.md) — scanning configuration
- [Scanner API](../scanner/) — triggering scans and watch mode
+26 -2
View File
@@ -21,9 +21,10 @@ Complete reference for Bookhoard REST API endpoints.
- [Conflicts](conflicts/) - Sync conflict resolution
- [Queue](queue/) - Sync queue management
- [Scanner](scanner/) - Library scanning and watch mode (admin)
- [System](system/) - Tunable system settings and configuration (admin)
- [OPDS](opds/) - Open Publication Distribution
- [KOReader](koreader/) - KOReader sync protocol
- [Kobo](kobo/) - Kobo sync protocol
- [Kobo](kobo/) - Kobo sync protocol (coming soon)
- [WebSocket](websocket/) - Real-time sync events
---
@@ -51,6 +52,8 @@ See [Admin Operations](admin/)
- GET /api/auth/users - List all users (admin)
- PUT /api/auth/users/:id/max-devices - Update user device limit (admin)
- GET /api/admin/hash-conflicts - List pending hash conflict groups (admin) — see [Hash Conflicts](admin/hash-conflicts.md)
- POST /api/admin/hash-conflicts/:id/resolve - Resolve a conflict (keep / keep_all) (admin)
## Users & Profiles
@@ -71,7 +74,10 @@ See [Library Management](libraries/)
- DELETE /api/libraries/:id/folders - Delete library folder (admin)
- GET /api/libraries/:id/stats - Get library statistics (admin)
- GET /api/libraries/:id/media-items - Get library media items (admin)
- GET /api/libraries/browse - Browse server directories (admin)
- POST /api/libraries/:id/scan - Scan library (admin)
- GET /api/libraries/scan-settings - Legacy scan settings (admin; superseded by /api/system/settings)
- PUT /api/libraries/scan-settings - Legacy scan settings update (admin; superseded by /api/system/settings)
- GET /api/libraries/visibility - Get visible libraries
- POST /api/libraries/visibility - Set library visibility
@@ -101,6 +107,10 @@ See [Media Item Operations](media-items/)
- GET /api/media-items/:id/highlights/:highlightId - Get highlight
- PUT /api/media-items/:id/highlights/:highlightId - Update highlight
- DELETE /api/media-items/:id/highlights/:highlightId - Delete highlight
- GET /api/media-items/:id/bookmarks - Get bookmarks
- GET /api/media-items/:id/annotations/deleted - List deleted annotations (history)
- POST /api/media-items/:id/annotations/:annotationId/restore - Restore a deleted annotation
- DELETE /api/media-items/:id/annotations/:annotationId?annotation_type=highlight|note|bookmark - Permanently delete a deleted annotation
- POST /api/media-items - Create media item (admin)
- PUT /api/media-items/:id - Update media item (admin)
- DELETE /api/media-items/:id - Delete media item (admin)
@@ -134,10 +144,21 @@ See [Device Registration & Sync](devices/)
- GET /api/devices/pending - List pending registrations (admin)
- GET /api/devices/approve/:registration_id - Approve registration (admin)
- POST /api/devices/reject/:registration_id - Reject registration (admin)
- POST /api/devices/:id/shelves - Add to shelf (Kobo)
- POST /api/devices/:id/shelves - Add to shelf (Kobo; used by native Kobo sync, coming soon)
- GET /api/devices/:id/shelves - Get shelf contents
- DELETE /api/devices/:id/shelves - Remove from shelf
- DELETE /api/devices/:id/shelves/clear - Clear shelf
- GET /api/devices/:id/sidecar - Get device sidecar config (.bookhoard.json) — see [Sidecar Config](devices/get_sidecar_config.md)
- GET /api/devices/:id/sidecar/download - Download sidecar config as a file
## System Settings & Configuration
See [System API](system/)
- GET /api/system/settings - List all tunable settings with metadata (admin)
- PUT /api/system/settings - Validate, persist, and reload a single setting (admin)
- GET /api/system/config - Raw key/value system configuration (admin)
- PUT /api/system/config - Update raw config values (admin)
## Analytics
@@ -220,12 +241,15 @@ See [OPDS Feeds](opds/)
See [KOReader Sync](koreader/) and [Sync Protocol](sync/koreader-protocol.md)
- POST /api/sync/koreader/progress - Sync reading progress
- GET /api/sync/koreader/resolve?sha256={hash} - Resolve a book UUID by file SHA-256
- GET /api/sync/koreader/metadata/:uuid - Get book metadata
- GET /api/sync/koreader/library - Get device library
- POST /api/sync/koreader/bookmarks - Sync bookmarks
## Kobo Sync Protocol
> **Status: Coming Soon** — Native Kobo sync is implemented server-side but not yet supported on real devices. These endpoints are under active development and may change.
See [Kobo Sync](kobo/) and [Sync Protocol](sync/kobo-protocol.md)
- POST /api/sync/kobo/markup - Sync markup highlights
+3 -1
View File
@@ -26,7 +26,7 @@ Authenticate with email and password.
```json
{
"token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
"access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
"refresh_token": "d4f5g6h7...",
"token_type": "Bearer",
"expires_in": 604800,
@@ -41,6 +41,8 @@ Authenticate with email and password.
}
```
Note: the access token field is `access_token` (not `token`). Nullable profile fields (`first_name`, `last_name`) may be empty strings.
**Set-Cookie Header**:
```
@@ -158,7 +158,7 @@ The frontend toast.js interceptor:
- **Backend**: Automatically manages HTTP-only cookie
- **Frontend**: Store tokens in localStorage for API calls
### Mobile Applications
### Mobile Applications (coming later)
- Store access token in secure storage (Keychain/Keystore)
- Store refresh token in secure storage
+13 -19
View File
@@ -2,7 +2,7 @@
Check device registration status or get device details.
**Endpoint**: `POST /api/devices/auth/status` or `GET /api/devices/{device_id}`
**Endpoint**: `POST /api/devices/register/status` or `GET /api/devices/{device_id}`
**Auth**: Not required for status check, Required for device details
**Content-Type**: `application/json` (for status check)
@@ -24,7 +24,9 @@ Check device registration status or get device details.
```json
{
"status": "pending|approved|expired",
"status": "pending|approved",
"message": "awaiting user approval",
"expires_in": 123,
"auth_token": "device-bearer-token...",
"device_id": "uuid",
"sync_endpoints": {
@@ -35,24 +37,16 @@ Check device registration status or get device details.
}
```
## Response (200 OK) - Device Details
`status` is `pending` or `approved`. While pending, the response includes `message` and `expires_in` (seconds remaining). Once approved, the response includes `auth_token`, `device_id`, and `sync_endpoints`; `auth_token` fields are empty when pending.
```json
{
"id": "uuid",
"device_name": "My Kobo Clara",
"device_type": "kobo",
"last_sync": "2026-01-31T10:00:00Z",
"last_seen": "2026-01-31T10:05:00Z",
"sync_enabled": true,
"auto_sync": true,
"sync_frequency_minutes": 5
}
```
**The approved response is single-use**: the registration is deleted from the pending map once returned, so store the `auth_token` immediately. A repeat status check for the same `registration_id` returns 404.
## Error Responses
| Code | Description |
| ---- | --------------------------------------------- |
| 401 | Invalid or expired token (for device details) |
| 404 | Device or registration not found |
| Code | Description |
| ---- | -------------------------------------------------- |
| 400 | Invalid or missing `registration_id` |
| 404 | Registration not found (unknown or already issued) |
| 410 | Registration expired (`{"error": "registration expired"}`) |
Note: expiration is signaled by HTTP 410 Gone, not a `"status": "expired"` value. Pending registrations are held in server memory, so a server restart also invalidates them (subsequent checks return 404).
@@ -0,0 +1,68 @@
# Get Device Sidecar Config
Returns the KOReader/Kobo sidecar configuration (`.bookhoard.json`) for a device: server endpoints, the user's books (keyed by SHA-256 with UUID fallback), and collections. Used by the Bookhoard KOReader plugin to self-configure after approval.
**Endpoint**: `GET /api/devices/{id}/sidecar`
**Auth**: User JWT (device owner or admin)
### Response (200 OK)
```json
{
"version": "1",
"bookhoard": {
"opds_catalog": "https://bookhoard.example.com/opds/devices/<device-id>/catalog",
"sync_api": "https://bookhoard.example.com/api/sync/kobo",
"opds_base_url": "https://bookhoard.example.com/opds",
"api_base_url": "https://bookhoard.example.com",
"device_id": "<device-id>",
"device_token": "dev_..."
},
"books": {
"abc123sha256...": {
"bookhoard_uuid": "media-item-uuid",
"title": "The Hobbit",
"author": "J. R. R. Tolkien",
"available_formats": ["epub", "kepub"],
"sha256": "abc123sha256...",
"file_path": "/books/hobbit.epub"
}
},
"collections": [
{ "name": "Favorites", "shelf_mapping": "Favorites" }
],
"opds_enabled": true,
"sidecar_enabled": true,
"last_updated": "2026-08-20T12:00:00Z"
}
```
**Notes**:
- The `books` map is keyed by per-format SHA-256 (falling back to the item UUID), so a book downloaded in a different format (e.g. KEPUB) still matches its primary entry. Each entry lists `available_formats` for the item.
- `available_formats` includes `kepub` when the source is an EPUB (conversion available).
### Example Request
```bash
curl https://bookhoard.example.com/api/devices/<device-id>/sidecar \
-H "Authorization: Bearer <token>"
```
---
# Download Device Sidecar Config
Generates the same configuration as a downloadable `.bookhoard.json` file for manual device setup.
**Endpoint**: `GET /api/devices/{id}/sidecar/download`
**Auth**: User JWT (device owner or admin)
### Response (200 OK)
**Headers**:
- `Content-Type`: `application/json`
- `Content-Disposition`: attachment; filename="<device-name>.bookhoard.json"
**Body**: the sidecar JSON (same shape as above).
@@ -28,14 +28,19 @@ Register a new device for sync.
```json
{
"device_id": "uuid",
"registration_id": "registration-uuid",
"auth_url": "https://bookhoard.com/devices/auth/confirm/abc123",
"auth_url": "https://bookhoard.com/devices/approve/abc123",
"qr_code": "data:image/png;base64,iVBORw0KG...",
"expires_in": 300
"expires_in": 300,
"poll_interval": 3,
"setup_instructions": {
"koreader": "Calibre URL: https://bookhoard.com/api/sync/koreader"
}
}
```
Open `auth_url` (or scan the QR code) while logged in to approve; the registration expires after 5 minutes. Poll `POST /api/devices/register/status` at `poll_interval` seconds until `status` is `approved`, at which point the response includes the device's `auth_token`, `device_id`, and `sync_endpoints`.
## Error Responses
| Code | Description |
@@ -1,5 +1,7 @@
# Analytics GetTests
> **Status: Coming Soon** — Native Kobo sync is not yet supported on real devices; this endpoint is under active development and may change.
Kobo analytics endpoint (device compatibility).
**Endpoint**: `POST /api/sync/kobo/v1/analytics/gettests`
+2
View File
@@ -1,5 +1,7 @@
# Bookmark Sync
> **Status: Coming Soon** — Native Kobo sync is not yet supported on real devices; this endpoint is under active development and may change.
Sync bookmarks from Kobo device.
**Endpoint**: `POST /api/sync/kobo/bookmark`
@@ -1,5 +1,7 @@
# Kobo Initialization
> **Status: Coming Soon** — Native Kobo sync is not yet supported on real devices; this endpoint is under active development and may change.
Initialize Kobo device sync.
**Endpoint**: `GET /api/sync/kobo/v1/initialization`
+2
View File
@@ -1,5 +1,7 @@
# Markup Sync
> **Status: Coming Soon** — Native Kobo sync is not yet supported on real devices; this endpoint is under active development and may change.
Sync markup highlights and annotations from Kobo device.
**Endpoint**: `POST /api/sync/kobo/markup`
@@ -1,5 +1,7 @@
# Sync From Server
> **Status: Coming Soon** — Native Kobo sync is not yet supported on real devices; this endpoint is under active development and may change.
Push content and metadata to Kobo device.
**Endpoint**: `POST /api/sync/kobo/sync-from-server`
@@ -0,0 +1,50 @@
# Resolve Book
Map a book's file SHA-256 to its Bookhoard UUID without touching progress
state. Used by devices to link a freshly downloaded book before their first
pull, so the device's first-page position is never pushed (which would
conflict with server-side progress for books already mid-read).
**Endpoint**: `GET /api/sync/koreader/resolve`
**Auth**: Required (Device authentication)
## Query Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ------------------------------------ |
| sha256 | string | Yes | File content hash (64 hex characters) |
Resolution is format-aware: the hash is checked against both
`media_items.file_sha256` and `media_item_formats.file_sha256`, so a
converted file (KEPUB/PDF) matches its media item too.
## Device Authentication
This endpoint requires device authentication (not user JWT). Devices
authenticate using their device credentials.
### Example Request
```http
GET /api/sync/koreader/resolve?sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: Bearer {device_token}
```
## Response (200 OK)
```json
{
"book_uuid": "550e8400-e29b-41d4-a716-446655440000",
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"title": "Book Title",
"author": "Author Name"
}
```
## Error Responses
| Code | Description |
| ---- | -------------------------------------------- |
| 400 | Missing or malformed `sha256` parameter |
| 401 | Device authentication failed |
| 404 | No book in the library matches the given hash |
+70 -36
View File
@@ -1,49 +1,81 @@
# Sync Bookmarks
Sync bookmarks from KOReader device.
Sync bookmarks, notes, and highlights from a KOReader device (bidirectional — the response also returns the server's current state for the book so the device can reconcile).
**Endpoint**: `POST /api/sync/koreader/bookmarks`
**Auth**: Required (Device authentication)
## Device Authentication
This endpoint requires device authentication (not user JWT). Devices authenticate using their device credentials.
**Auth**: Device token (Bearer)
## Request Body
| Field | Type | Required | Description |
| --------- | ------------- | -------- | ------------------------- |
| device_id | string (UUID) | Yes | Device UUID |
| bookmarks | array | Yes | Array of bookmark objects |
| Field | Type | Required | Description |
| ------------ | ------ | --------------------- | ----------------------------------------------------------------- |
| book_uuid | string | one of uuid/sha | Book UUID (highest-confidence match) |
| book_sha256 | string | one of uuid/sha | Full-file SHA-256 (64 hex chars); format-aware (also matches `media_item_formats`, so a KEPUB/PDF download matches) |
| bookmarks | array | No | Bookmark objects |
| notes | array | No | Note objects |
| highlights | array | No | Highlight objects |
### Bookmark Object
At least one of `book_uuid` or `book_sha256` is required; `book_sha256` resolves through the shared BookResolver.
| Field | Type | Required | Description |
| ---------------- | ------- | -------- | -------------------------- |
| book | string | Yes | Book identifier |
| chapter | string | No | Chapter title |
| page | integer | No | Page number |
| position | float | Yes | Position in document (0-1) |
| notes | string | No | Bookmark notes |
| highlighted_text | string | No | Highlighted text |
| time | string | Yes | ISO 8601 timestamp |
| created_at | string | Yes | ISO 8601 timestamp |
### Bookmark / Note / Highlight Object
All three types share the same KOReader annotation shape:
| Field | Type | Required | Description |
| ------------ | ------- | -------- | ---------------------------------------------------- |
| chapter | int | No | Chapter index |
| datetime | string | No | ISO 8601 creation/edit timestamp |
| pos0 / pos1 | string | No | Start/end xpointer (or `page:N` / bare page) |
| page | int | No | Page number (fallback location when `pos0` is empty) |
| text | string | No | Highlighted text |
| notes | string | No | Note text attached to the annotation |
| type | string | No | Annotation type (`highlight`, `note`, `bookmark`) |
| color | string | No | Highlight color (highlights only) — KOReader palette name, see below |
| percentage | float | No | Position within the book (0-1) |
| book_sha256 | string | No | Per-annotation SHA-256; overrides the request-level book match |
| dedup_key | string | No | Stable echo key; an entry whose content is unchanged from what the server previously served is recognized as an echo rather than a new edit |
### Color Semantics
KOReader paints highlights from a fixed palette of color names; the web reader uses hex swatches. Colors are mapped at the boundary (unmappable values fall back to yellow on both sides):
| KOReader name | Web hex |
| ------------- | --------- |
| yellow, orange | `#ffd54f` |
| green, olive | `#a5d6a7` |
| cyan, blue | `#90caf9` |
| purple | `#ce93d8` |
| red | `#f48fb1` |
- An echo (device re-reporting an annotation it received from the server) carries **no color**, so the stored web color is never clobbered.
- A non-empty color means the user edited the highlight on the device; it is mapped to the nearest web swatch.
### Example Request
```json
{
"device_id": "550e8400-e29b-41d4-a716-446655440000",
"book_sha256": "64-hex-char-sha256",
"bookmarks": [
{
"book": "book.epub",
"chapter": "Chapter 1",
"chapter": 3,
"datetime": "2026-08-20T10:00:00Z",
"pos0": "/body/Doc[4]/Sec[2]",
"page": 25,
"position": 0.125,
"notes": "Important section",
"highlighted_text": "Text to remember",
"time": "2026-02-08T10:00:00Z",
"created_at": "2026-02-08T10:00:00Z"
"text": "",
"type": "bookmark",
"percentage": 0.125
}
],
"highlights": [
{
"datetime": "2026-08-20T10:05:00Z",
"pos0": "/body/Doc[4]/Sec[2]/text()[3]:0",
"pos1": "/body/Doc[4]/Sec[2]/text()[3]:42",
"text": "Text to remember",
"notes": "Why this matters",
"type": "highlight",
"color": "blue",
"dedup_key": "echo-key-from-server"
}
]
}
@@ -53,15 +85,17 @@ This endpoint requires device authentication (not user JWT). Devices authenticat
```json
{
"message": "Bookmarks synced successfully",
"synced_count": 1
"sync_status": "ok",
"bookmarks_synced": 1,
"notes_synced": 0,
"highlights_synced": 1
}
```
## Error Responses
| Code | Description |
| ---- | ---------------------------- |
| 401 | Device authentication failed |
| 400 | Invalid request data |
| 404 | Device not found |
| Code | Description |
| ---- | -------------------------------------------------- |
| 400 | Invalid request, or neither uuid nor SHA provided |
| 401 | Missing/invalid device token |
| 404 | Book not found by SHA-256 |
@@ -2,7 +2,7 @@
Retrieve all libraries visible to the current user.
**Endpoint**: `GET /api/libraries/visible`
**Endpoint**: `GET /api/libraries/visibility`
**Auth**: Required
## Request Headers
@@ -14,26 +14,33 @@ Retrieve all libraries visible to the current user.
### Example Request
```http
GET /api/libraries/visible
GET /api/libraries/visibility
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
```
## Response (200 OK)
A top-level JSON **array** of library rows:
```json
{
"libraries": [
{
"id": "uuid",
"name": "My Ebooks",
"description": "Ebook collection",
"type_name": "ebooks",
"is_visible": true
}
]
}
[
{
"id": "uuid",
"name": "My Ebooks",
"description": "Ebook collection",
"library_type_id": "uuid",
"created_by_admin_id": "uuid",
"created_at": "2026-01-31T10:00:00Z",
"updated_at": "2026-01-31T10:00:00Z",
"type_name": "ebooks",
"type_description": "Ebook libraries",
"is_visible": true
}
]
```
Nullable columns (`description`, `type_description`) serialize as `null` when unset. Timestamps are RFC 3339.
## Error Responses
| Code | Description |
@@ -0,0 +1,84 @@
# Deleted Annotations History
List, restore, or permanently delete tombstoned annotations (highlights,
notes, bookmarks) for a book. Deletions — from the web or propagated from a
synced device — are soft-deleted and retained for the sync retention window
(default 30 days), powering the book page's "Recently deleted" list. A
restore returns the row to the active set on every synced device; a purge
removes it immediately and irreversibly.
All endpoints require user JWT authentication and operate only on the
caller's own annotations.
## List Deleted Annotations
**Endpoint**: `GET /api/media-items/:id/annotations/deleted`
Returns tombstoned annotations for the book, newest deletion first.
### Response (200 OK)
```json
{
"deleted_annotations": [
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"annotation_type": "highlight",
"display_text": "the chosen text",
"secondary_text": "user note",
"color": "#ffd54f",
"deleted_at": "2026-08-22T15:04:05Z",
"created_at": "2026-08-01T10:00:00Z"
}
],
"total": 1
}
```
| Field | Description |
| --------------- | ------------------------------------------------------ |
| annotation_type | `highlight`, `note`, or `bookmark` |
| display_text | Highlighted text / note content / bookmark title |
| secondary_text | Note text (highlights) or notes field (bookmarks) |
## Restore Deleted Annotation
**Endpoint**: `POST /api/media-items/:id/annotations/:annotationId/restore`
Body (or query param) `annotation_type` must be `highlight`, `note`, or
`bookmark`. Clears the tombstone; the annotation reappears in the active
set and re-syncs to devices on their next pull.
```json
{ "annotation_type": "highlight" }
```
### Response (200 OK)
```json
{ "restored": true }
```
404 when no matching *deleted* annotation exists for this user and book.
## Permanently Delete Annotation
**Endpoint**: `DELETE /api/media-items/:id/annotations/:annotationId?annotation_type=highlight|note|bookmark`
Removes the tombstoned row from the history immediately. Irreversible —
unlike the tombstone itself, which is restorable until the retention window
lapses and the daily maintenance sweep purges it.
### Response (200 OK)
```json
{ "purged": true }
```
## Error Responses
| Code | Description |
| ---- | -------------------------------------------------- |
| 400 | Invalid IDs or missing/unknown `annotation_type` |
| 401 | Not authenticated |
| 404 | No matching deleted annotation |
@@ -1,17 +1,22 @@
# List Media Items
Retrieve a paginated list of media items from a library.
Retrieve a paginated list of media items, scoped to a library or across all libraries.
**Endpoint**: `GET /api/media-items`
**Auth**: Required
## Query Parameters
| Parameter | Type | Required | Description |
| ---------- | ------- | -------- | ----------------------------------------------- |
| library_id | string | Yes | Library UUID |
| limit | integer | No | Number of items to return (max 100, default 20) |
| offset | integer | No | Number of items to skip |
| Parameter | Type | Required | Description |
| ---------- | ------ | -------- | ------------------------------------------------------ |
| library_id | string | No | Library UUID. If omitted, items from all libraries are returned |
| limit | int | No | Items to return (default 50, max 1000) |
| offset | int | No | Items to skip (must be >= 0) |
| sort | string | No | Sort expression, default `created_at DESC` |
### Allowed sort expressions
`created_at`, `title`, `author`, `series`, `date_published`, `copyright_year`, `page_count`, `genre` — each with ` ASC` or ` DESC` (e.g. `title ASC`). Any other value silently falls back to `created_at DESC`.
## Request Headers
@@ -22,46 +27,121 @@ Retrieve a paginated list of media items from a library.
### Example Request
```http
GET /api/media-items?library_id=uuid&limit=20&offset=0
GET /api/media-items?library_id=uuid&limit=20&offset=0&sort=title%20ASC
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
```
## Response (200 OK)
The response body is `{"data": [...]}` in both modes. The item shape differs by mode.
**No total is returned** — page until fewer items than `limit` come back.
### With `library_id` — full database rows
Nullable columns serialize as `null`.
```json
{
"media_items": [
"data": [
{
"id": "uuid",
"library_id": "uuid",
"title": "Book Title",
"author": "Author Name",
"isbn": "978-...",
"description": "Book description",
"file_path": "/path/to/book.epub",
"file_path": "relative/path/book.epub",
"file_size": 1024000,
"mime_type": "application/epub+zip",
"cover_image_path": "/path/to/cover.jpg",
"cover_image_path": "relative/path/cover.jpg",
"series": "Series Name",
"series_number": 1,
"tags": ["sci-fi", "space opera"],
"tags_search": ["sci fi", "space opera"],
"contributors": ["Author Name", "ACME CORP."],
"contributors_search": ["author name", "acme corp"],
"tags": ["sci-fi"],
"asin": null,
"date_published": "2023-06-01",
"publisher": null,
"contributors": ["Author Name"],
"language": "en",
"edition": null,
"page_count": 350,
"genre": "Science Fiction",
"copyright_year": 2023,
"created_at": "2026-01-31T10:00:00Z"
"goodreads_id": null,
"openlibrary_id": null,
"google_books_id": null,
"added_by_admin_id": "uuid",
"created_at": "2026-01-31T10:00:00Z",
"imported_at": "2026-01-31T10:00:00Z",
"updated_at": "2026-01-31T10:00:00Z",
"format_group": "epub",
"format_mimetype": "application/epub+zip",
"is_reflowable": true,
"has_fixed_layout": false,
"total_characters": 480000,
"chapter_count": 24
}
],
"total": 100
]
}
```
Note: in this mode `file_path` and `cover_image_path` are the raw relative storage paths, not URLs.
### Without `library_id` — curated items with resolved URLs
Across all libraries; file and cover paths are resolved to fetchable URL paths (`/uploads/...` or library-scoped paths):
```json
{
"data": [
{
"id": "uuid",
"library_id": "uuid",
"title": "Book Title",
"author": "Author Name",
"isbn": "978-...",
"description": "Book description",
"file_path": "/api/libraries/<uuid>/files/...",
"file_size": 1024000,
"mime_type": "application/epub+zip",
"cover_image_path": "/api/libraries/<uuid>/files/.../cover.jpg",
"series": "Series Name",
"series_number": 1,
"tags": ["sci-fi"],
"asin": null,
"date_published": "2023-06-01",
"publisher": null,
"contributors": ["Author Name"],
"language": "en",
"edition": null,
"page_count": 350,
"genre": "Science Fiction",
"created_at": "2026-01-31T10:00:00Z",
"updated_at": "2026-01-31T10:00:00Z",
"format_group": "epub",
"manga_type": null,
"reading_direction": null,
"series_count": null,
"volume": null,
"imprint": null,
"age_rating": null,
"web_url": null,
"metadata_notes": null,
"community_rating": null,
"story_arc": null,
"is_black_and_white": false,
"alternate_info": null,
"scan_information": null,
"summary": null
}
]
}
```
## Error Responses
| Code | Description |
| ---- | ----------------------------------------- |
| 400 | Invalid query parameters |
| 401 | Invalid or expired token |
| 403 | User does not have access to this library |
| Code | Description |
| ---- | ------------------------------------------ |
| 400 | Invalid `library_id`, `offset` < 0 |
| 401 | Invalid or expired token |
| 500 | Query failure (returned as `{"error": …}`) |
+2
View File
@@ -1,5 +1,7 @@
# Kobo Sync Protocol
> **Status: Coming Soon** — Native Kobo sync is implemented server-side but not yet supported on real devices. These endpoints are under active development and may change. Until then, KOReader (which runs on Kobo hardware) is fully supported.
Kobo uses a proprietary sync protocol with JSON payloads.
## Kobo Markup Sync
+102 -14
View File
@@ -17,20 +17,31 @@ KOReader uses a custom JSON-based sync protocol.
### Request Body
| Field | Type | Required | Description |
| ------------------ | ------- | -------- | ----------------------------- |
| library_id | string | No | Library UUID |
| books | array | Yes | Array of book sync data |
| books[].uuid | string | Yes | Book UUID |
| books[].title | string | Yes | Book title |
| books[].authors | array | Yes | Array of author names |
| books[].progress | float | Yes | Progress percentage (0-1) |
| books[].percentage | float | Yes | Progress percentage (0-1) |
| books[].last_read | string | Yes | ISO 8601 timestamp |
| books[].chapter | integer | No | Current chapter |
| books[].epubcfi | string | No | EPUB CFI location |
| books[].character | integer | No | Character offset |
| books[].bookmarks | array | No | Array of bookmarks/highlights |
| Field | Type | Required | Description |
| ------------------ | ------- | -------- | ---------------------------------------------------- |
| library_id | string | No | Library UUID |
| books | array | Yes | Array of book sync data |
| books[].uuid | string | No\* | Book UUID (highest-confidence match; omitted on first sync of a newly downloaded book) |
| books[].sha256 | string | No\* | Full-file SHA-256 (64 hex chars); used to resolve the book when `uuid` is absent |
| books[].file_path | string | No | Device-local file path; used to create/look up a device file alias |
| books[].title | string | Yes | Book title |
| books[].authors | array | Yes | Array of author names |
| books[].progress | float | Yes | Progress percentage (0-1) |
| books[].percentage | float | Yes | Progress percentage (0-1) |
| books[].last_read | string | Yes | ISO 8601 timestamp |
| books[].chapter | integer | No | Current chapter |
| books[].epubcfi | string | No | EPUB CFI location |
| books[].character | integer | No | Character offset |
| books[].bookmarks | array | No | Array of bookmarks/highlights (shape, color mapping, and echo/dedup rules: see [Sync Bookmarks](../koreader/sync_bookmarks.md)) |
| books[].deleted_highlights | array | No | Highlights deleted on the device: `[{ "dedup_key": "..." }]` — keys previously served to this device (see [Deletion propagation](#deletion-propagation)) |
| books[].deleted_bookmarks | array | No | Bookmarks deleted on the device: `[{ "dedup_key": "..." }]` |
\* At least one of `uuid` or `sha256` should be present. The server resolves the
book through the shared `BookResolver` with this priority: `uuid``sha256`
`file_path` alias → `title`/`author`. SHA-256 matching is **format-aware**: it
checks `media_items.file_sha256` first, then `media_item_formats.file_sha256`, so
a converted file (e.g. KEPUB or PDF) downloaded via OPDS matches even though its
hash differs from the primary format's hash.
### Example Request
@@ -85,6 +96,41 @@ KOReader uses a custom JSON-based sync protocol.
}
```
## Book Resolution (UUID lookup)
**Endpoint**: `GET /api/sync/koreader/resolve?sha256={hash}`
**Auth**: Device token required
Read-only lookup mapping a file SHA-256 to the book's UUID (format-aware,
same `BookResolver` path as the progress push). Devices call this on the
first open of a newly downloaded book to learn the UUID **before** their
first pull. Full details: [Resolve Book](../koreader/resolve_book.md).
This matters for conflict avoidance: a device that pushes to bootstrap its
identity transmits its current (first-page) position, which the server
treats as a real progress update — overwriting/conflicting with genuine
mid-read progress from other sources. Resolve, then pull, then push.
### Example Request
```http
GET /api/sync/koreader/resolve?sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: Bearer device-token
```
### Response (200 OK)
```json
{
"book_uuid": "book-uuid",
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"title": "Book Title",
"author": "Author Name"
}
```
404 when no book in the library matches the hash.
## KOReader Metadata Fetch
**Endpoint**: `GET /api/sync/koreader/metadata/{book_uuid}`
@@ -102,6 +148,7 @@ Authorization: Bearer device-token
```json
{
"uuid": "book-uuid",
"sha256": "ff3e4501bf9d72dea2ae28731a6cb5b83d7a7532c05b5d2dd083d0dbc9193ebf",
"title": "Book Title",
"authors": ["Author Name"],
"progress": {
@@ -119,3 +166,44 @@ Authorization: Bearer device-token
"last_sync": "2026-01-30T20:00:00Z"
}
```
`sha256` is the canonical primary-format hash of the book on the server. It is
returned so clients can cache it regardless of how the book was originally
obtained. The library list endpoint (`GET /api/sync/koreader/library`) includes
the same `sha256` field on each book.
## Deletion propagation
The progress push is upsert-only: absence of an annotation from
`highlights`/`notes`/`bookmarks` is **never** interpreted as a delete (a
client with a category disabled must not wipe the server). Deletions are
reported explicitly:
- Devices remember the `dedup_key` of every annotation the server served
them (persisted locally, e.g. KOReader's sidecar `bookhoard_known_keys`).
- When one of those annotations no longer exists locally, the next push
lists its key in `deleted_highlights` / `deleted_bookmarks`.
- The server tombstones the matching rows (`deleted = TRUE`, kept for the
retention window). Tombstones are served back to *other* devices via the
metadata fetch's `deleted_highlights` / `deleted_bookmarks` arrays so the
deletion converges everywhere.
- A stale replay pushing the annotation's content cannot resurrect the
tombstone: device pushes carry no modification timestamp, so the save is
treated as older than the delete.
- Restoring is possible from the web book page's deleted-annotation
history (`GET /api/media-items/:id/annotations/deleted`, restore/purge
endpoints) until the retention window lapses.
Because keys are only learned from server pulls, a device-native annotation
deleted locally is simply never pushed again — it can never be mis-flagged
as a server annotation deletion.
## Book identification
Every client/sync interface (KOReader, Kobo, OPDS, the device-link UI, and any
future mobile app) resolves books through a single shared service:
[`internal/services/book_resolver.go`](../../../internal/services/book_resolver.go).
The import-time SHA-256 (stored on `media_items.file_sha256`, plus a per-format
hash on `media_item_formats.file_sha256` for KEPUB/PDF) is the canonical shared
identifier. New clients should resolve by SHA-256 via `BookResolver` rather than
re-implementing their own matcher.
+77
View File
@@ -0,0 +1,77 @@
# System Config API
## Overview
Raw key/value system configuration storage (backed by the `system_config` table). Unlike the typed [System Settings API](settings.md), this endpoint reads and writes arbitrary config keys as plain strings — including keys without registry metadata, such as `base_url`.
**Base URL**: `/api/system`
**Authentication**: Admin JWT token required
**Content-Type**: `application/json`
---
## Endpoints
### Get System Configuration
Retrieve all system configuration entries as a flat key/value map.
**Endpoint**: `GET /api/system/config`
**Authentication**: Admin role required
**Response**: **200 OK**
```json
{
"base_url": "http://192.168.1.100:8765",
"default_timezone": "America/New_York"
}
```
**Example**:
```bash
curl -X GET https://bookhoard.example.com/api/system/config \
-H "Authorization: Bearer <admin_token>"
```
---
### Update System Configuration
Update one or more config values.
**Endpoint**: `PUT /api/system/config`
**Authentication**: Admin role required
**Request Body**: a flat map of keys to string values. Only the supplied keys are updated.
```json
{
"base_url": "https://bookhoard.example.com"
}
```
**Validation**: values for known keys are validated where applicable — for example, `default_timezone` must be a valid IANA timezone (`time.LoadLocation`); invalid values return `400` without persisting.
**Response**: **200 OK** on success; `400` (invalid value/format), `401`, `403`, `500` on failure.
**Example**:
```bash
curl -X PUT https://bookhoard.example.com/api/system/config \
-H "Authorization: Bearer <admin_token>" \
-H "Content-Type: application/json" \
-d '{"base_url": "https://bookhoard.example.com"}'
```
> **Note:** settings that appear in the typed settings registry (e.g. `default_timezone`) are better managed through [`PUT /api/system/settings`](settings.md), which also returns metadata and reload hints. Writes through either endpoint refresh the shared registry cache.
---
## Related Endpoints
- [System Settings API](settings.md) — typed, validated tunable settings with metadata
- `GET /api/devices/:id/sidecar` — device setup config derived from system config (see [Devices API](../devices/))
+116 -109
View File
@@ -1,10 +1,10 @@
# System Scan Settings API
# System Settings API
## Overview
The System Scan Settings API allows administrators to configure system-wide scan settings that apply to all libraries. These settings control the automatic scanning behavior for the entire Bookhoard system.
The System Settings API is the canonical way to read and write Bookhoard's tunable system settings (scanning, security, rate limits, sync, performance, and defaults). Every setting carries full metadata — type, range, category, description, and whether a restart is required — so the admin UI (and API clients) can render and validate settings generically.
**Base URL**: `/api/libraries`
**Base URL**: `/api/system`
**Authentication**: Admin JWT token required
**Content-Type**: `application/json`
@@ -12,49 +12,61 @@ The System Scan Settings API allows administrators to configure system-wide scan
## Endpoints
### Get System Scan Settings
### List All Settings
Retrieve the current system-wide scan settings.
Retrieve every known tunable setting with its current value and metadata.
**Endpoint**: `GET /api/libraries/scan-settings`
**Endpoint**: `GET /api/system/settings`
**Authentication**: Admin role required
**Response**:
- **200 OK**: Returns current scan settings
- **401 Unauthorized**: Invalid or missing authentication
- **403 Forbidden**: User does not have admin role
- **500 Internal Server Error**: Server error
**Response Body**:
**Response**: **200 OK**
```json
{
"scan_poll_interval_seconds": 60,
"auto_scan_enabled": true
}
[
{
"key": "scan_poll_interval_seconds",
"value": "60",
"type": "int",
"min": "1",
"max": "3600",
"requires_restart": false,
"category": "scanner",
"group": "Scanning",
"description": "How often to scan all libraries (seconds)",
"is_default": true
}
]
```
**Fields**:
**Entry fields**:
- `scan_poll_interval_seconds` (integer): How often to poll for file changes in seconds (1-3600)
- `auto_scan_enabled` (boolean): Whether auto-scanning is enabled system-wide
| Field | Type | Description |
| ------------------ | ------- | -------------------------------------------------------- |
| `key` | string | Setting identifier (stable API name) |
| `value` | string | Current value (validated/clamped by the registry) |
| `type` | string | `int`, `bool`, or `string` |
| `min` / `max` | string | Range bounds for `int` settings (omitted otherwise) |
| `requires_restart` | boolean | Change takes effect only after a server restart |
| `category` | string | Coarse area: `scanner`, `security`, `api`, `sync`, `performance`, `general` |
| `group` | string | Sub-section shown in the admin UI |
| `description` | string | Human-readable description |
| `is_default` | boolean | True when the current value equals the compiled default |
**Example**:
```bash
curl -X GET https://bookhoard.example.com/api/libraries/scan-settings \
curl -X GET https://bookhoard.example.com/api/system/settings \
-H "Authorization: Bearer <admin_token>"
```
---
### Update System Scan Settings
### Update a Setting
Update the system-wide scan settings.
Validate, persist, and reload a single setting.
**Endpoint**: `PUT /api/libraries/scan-settings`
**Endpoint**: `PUT /api/system/settings`
**Authentication**: Admin role required
@@ -62,128 +74,123 @@ Update the system-wide scan settings.
```json
{
"scan_poll_interval_seconds": 30,
"auto_scan_enabled": true
"key": "scan_poll_interval_seconds",
"value": "30"
}
```
**Fields**:
| Field | Type | Required | Description |
| ------- | ------ | -------- | ------------------------------- |
| `key` | string | Yes | Setting key (from the list) |
| `value` | string | Yes | New value, as a string |
- `scan_poll_interval_seconds` (integer, required): How often to poll for file changes in seconds
- Minimum: 1 (1 second)
- Maximum: 3600 (1 hour)
- Default: 60
- `auto_scan_enabled` (boolean, required): Whether auto-scanning is enabled system-wide
- Default: true
**Response**:
- **200 OK**: Settings updated successfully
- **400 Bad Request**: Invalid request parameters
- **401 Unauthorized**: Invalid or missing authentication
- **403 Forbidden**: User does not have admin role
- **500 Internal Server Error**: Server error
**Success Response Body**:
**Response**: **200 OK**
```json
{
"scan_poll_interval_seconds": 30,
"auto_scan_enabled": true,
"message": "scan settings updated successfully"
"key": "scan_poll_interval_seconds",
"value": "30",
"type": "int",
"min": "1",
"max": "3600",
"requires_restart": false,
"category": "scanner",
"group": "Scanning",
"description": "How often to scan all libraries (seconds)",
"is_default": false,
"reload_required": false,
"message": ""
}
```
**Error Response Body**:
- `reload_required: true` means the change takes effect only after a restart (e.g. rate limits, worker pool, lockout settings).
- Validation is type-aware: `int` values are checked against `min`/`max`, `bool` values must parse, `default_timezone` must be a valid IANA timezone via `time.LoadLocation`, and strings must be non-empty.
```json
{
"error": "error message"
}
```
**Validation Rules**:
- `scan_poll_interval_seconds` must be between 1 and 3600 seconds (1 second to 1 hour)
- Both fields are required
**Errors**: `400` (unknown key, invalid value, out of range), `401`, `403`, `503` (settings registry not initialized).
**Example**:
```bash
curl -X PUT https://bookhoard.example.com/api/libraries/scan-settings \
curl -X PUT https://bookhoard.example.com/api/system/settings \
-H "Authorization: Bearer <admin_token>" \
-H "Content-Type: application/json" \
-d '{
"scan_poll_interval_seconds": 30,
"auto_scan_enabled": true
}'
-d '{"key": "scan_poll_interval_seconds", "value": "30"}'
```
---
## Behavior
## Setting Catalog
### Poll Interval
Current tunable settings by category:
The `scan_poll_interval_seconds` setting determines how often the system will poll library folders for file changes as a fallback to real-time file watching.
**Scanner** (`scanner`)
**Constraints**:
| Key | Default | Range | Restart | Description |
| ------------------------------ | ------- | -------- | ------- | ----------------------------------------- |
| `scan_poll_interval_seconds` | `60` | 1-3600 | No | How often to scan all libraries (seconds) |
| `auto_scan_enabled` | `true` | - | No | Whether auto-scanning is enabled |
- Minimum: 1 second
- Maximum: 3600 seconds (1 hour)
- Default: 60 seconds
**General** (`general`)
### Auto-Scan Toggle
| Key | Default | Restart | Description |
| ----------------- | ------- | ------- | ------------------------- |
| `default_timezone`| `UTC` | No | System default timezone |
The `auto_scan_enabled` setting acts as a master switch for automatic scanning:
**Security** (`security`)
- When `true`: File watching and polling fallback are active for all libraries
- When `false`: No automatic file monitoring occurs (manual scans still available)
| Key | Default | Range | Restart | Description |
| ---------------------------- | --------- | ------------ | ------- | ---------------------------------------------- |
| `session_duration_seconds` | `604800` | 300-31536000 | No | How long a login session stays valid |
| `password_min_length` | `8` | 1-128 | No | Minimum password length |
| `password_require_upper` | `true` | - | No | Require at least one uppercase letter |
| `password_require_lower` | `true` | - | No | Require at least one lowercase letter |
| `password_require_number` | `true` | - | No | Require at least one number |
| `password_require_special` | `true` | - | No | Require at least one special character |
| `auth_rate_limit_per_min` | `10` | 1-10000 | **Yes** | Global auth API rate limit (req/min) |
| `login_max_attempts` | `5` | 1-100 | **Yes** | Failed login attempts before lockout |
| `login_lockout_minutes` | `15` | 1-10080 | **Yes** | Lockout duration after failed logins |
### File Watching System
**API** (`api`)
The scan settings control the file watching system which consists of:
| Key | Default | Range | Restart | Description |
| ------------------------------- | ------- | --------- | ------- | ------------------------------------ |
| `opds_default_page_size` | `50` | 1-500 | No | Default OPDS page size |
| `opds_max_page_size` | `200` | 1-1000 | No | Maximum OPDS page size |
| `device_rate_sync_per_min` | `60` | 1-10000 | No | Device sync requests per minute |
| `device_rate_progress_per_min` | `120` | 1-10000 | No | Device progress requests per minute |
| `device_rate_metadata_per_min` | `30` | 1-10000 | No | Device metadata requests per minute |
1. **Real-time file watching**: Uses fsnotify to detect file changes immediately
2. **Polling fallback**: If file watching fails or is unavailable, polls folders at the configured interval
**Sync** (`sync`)
The system applies these settings to all configured libraries automatically on startup.
| Key | Default | Range | Restart | Description |
| ------------------------------- | ------- | -------- | ------- | -------------------------------------------------- |
| `annotation_tombstone_ttl_days` | `30` | 1-3650 | No | How long deleted annotations are kept before purge |
| `sync_queue_interval_seconds` | `5` | 1-3600 | **Yes** | How often the sync queue flushes |
| `sync_queue_batch_size` | `50` | 1-10000 | **Yes** | Max items processed per sync queue flush |
**Performance** (`performance`)
| Key | Default | Range | Restart | Description |
| ------------------------ | ------- | --------- | ------- | ------------------------------------------- |
| `conversion_cache_ttl_hours` | `24` | 1-720 | No | How long converted (KEPUB) files are cached |
| `worker_pool_size` | `3` | 1-100 | **Yes** | Number of background worker goroutines |
| `worker_queue_cap` | `100` | 1-10000 | **Yes** | Background worker job queue capacity |
---
## Error Codes
## Legacy Scan Settings Routes
| Status Code | Error Description |
| ----------- | ---------------------------------------------------------- |
| 400 | Invalid request parameters (e.g., frequency outside range) |
| 401 | Missing or invalid JWT token |
| 403 | User lacks admin role |
| 500 | Internal server error (e.g., database connection issue) |
The older JSON routes still work for backward compatibility and now refresh the settings registry cache on write, but they are **superseded** by `GET/PUT /api/system/settings`:
- `GET /api/libraries/scan-settings` — returns only `scan_poll_interval_seconds` and `auto_scan_enabled`
- `PUT /api/libraries/scan-settings` — accepts `{ "scan_poll_interval_seconds": int, "auto_scan_enabled": bool }`
Both fields are backed by the same registry entries documented above.
---
## Related Endpoints
- `POST /api/libraries/{id}/scan` - Manually trigger a scan for a specific library (admin only)
- `GET /api/libraries` - List all libraries
- `GET /api/libraries/{id}` - Get details for a specific library
---
## Migration Notes
This API has been updated to use a new polling-based scanning system. The following changes were made:
- **Changed**: `scan_frequency_minutes` renamed to `scan_poll_interval_seconds`
- **Changed**: Unit changed from minutes to seconds (15-1440 minutes → 1-3600 seconds)
- **Removed**: Old scheduler-based scanning system
- **Added**: Real-time file watching with polling fallback
- **Preserved**: API endpoint paths remain the same
The new system ensures that:
1. File changes are detected in real-time when possible (via fsnotify)
2. Polling fallback catches missed events at the configured interval
3. Settings apply to all libraries system-wide
4. Only administrators can modify scan settings
5. The `auto_scan_enabled` setting controls both file watching and polling
- `GET/PUT /api/system/config` — raw key/value system configuration (see [System Config API](config.md))
- `POST /api/scanner/scan` — trigger a manual scan (see [Scanner API](../scanner/))
- `GET /api/admin/hash-conflicts` — duplicates found during hashing (see [Hash Conflicts API](../admin/hash-conflicts.md))
+14 -14
View File
@@ -11,8 +11,8 @@ Complete guide to Bookhoard documentation. Find what you need quickly.
**[User Documentation Portal](user/user-guide.md)** - Guides for using Bookhoard features
- **Device Setup**
- [Kobo Setup Guide](user/devices/kobo-setup.md) - Complete Kobo e-reader configuration
- [KOReader Setup Guide](user/devices/koreader-setup.md) - KOReader on Kindle/Kobo/PocketBook
- [KOReader Setup Guide](user/devices/koreader-setup.md) - KOReader on Kindle/Kobo/PocketBook hardware
- [Kobo Setup Guide](user/devices/kobo-setup.md) - Native Kobo sync (coming soon; use KOReader today)
- **Sync Configuration**
- [Universal Sync Guide](user/sync-guide.md) - Understanding sync, book matching, conflicts
@@ -38,12 +38,12 @@ Complete guide to Bookhoard documentation. Find what you need quickly.
- [Queue API](developer/api/queue/) - Sync queue management endpoints
- [Scanner API](developer/api/scanner/) - Library scanning and automated watch mode (admin)
- [KOReader API](developer/api/koreader/) - KOReader sync protocol endpoints
- [Kobo API](developer/api/kobo/) - Kobo sync protocol endpoints
- [Kobo API](developer/api/kobo/) - Kobo sync protocol endpoints (feature coming soon)
- [WebSocket API](developer/api/websocket/) - Real-time sync events
- **Protocol Specifications**
- [Kobo Sync Protocol](developer/api/sync/kobo-protocol.md) - Kobo device sync
- [KOReader Sync Protocol](developer/api/sync/koreader-protocol.md) - KOReader sync
- [Kobo Sync Protocol](developer/api/sync/kobo-protocol.md) - Kobo device sync (coming soon)
- [WebSocket API](developer/websocket-api.md) - Real-time events
### 🔧 For Operations
@@ -62,8 +62,8 @@ Complete guide to Bookhoard documentation. Find what you need quickly.
**[Contributing Portal](contributing/contributing.md)** - Development workflow
- [Development Guide](contributing/Development.md) - Architecture, setup, testing
- [PROJECT_GUIDELINES.md](PROJECT_GUIDELINES.md) - Development rules and standards
- [Development Guide](contributing/development.md) - Architecture, setup, testing
- [../PROJECT_GUIDELINES.md](../PROJECT_GUIDELINES.md) - Development rules and standards
---
@@ -75,7 +75,7 @@ Complete guide to Bookhoard documentation. Find what you need quickly.
| **Set up a device** | [User Portal → Device Setup](user/user-guide.md) |
| **Use the API** | [Developer Portal → API Docs](developer/development.md) |
| **Deploy Bookhoard** | [Operations Portal → Troubleshooting](operations/troubleshooting.md) |
| **Contribute code** | [Contributing Portal → Development Guide](contributing/Development.md) |
| **Contribute code** | [Contributing Portal → Development Guide](contributing/development.md) |
| **Understand sync** | [User Portal → Sync Guide](user/sync-guide.md) |
---
@@ -87,13 +87,13 @@ Complete guide to Bookhoard documentation. Find what you need quickly.
| Question | Answer |
| --------------------------- | ------------------------------------------------------ |
| ...install Bookhoard? | [README.md](../README.md) - Quick Start |
| ...set up my Kobo? | [Kobo Setup Guide](user/devices/kobo-setup.md) |
| ...set up KOReader? | [KOReader Setup Guide](user/devices/koreader-setup.md) |
| ...use a Kobo? | [Kobo Setup Guide](user/devices/kobo-setup.md) - native sync coming soon; KOReader works today |
| ...understand sync? | [Sync Guide](user/sync-guide.md) |
| ...resolve conflicts? | [Sync Guide](user/sync-guide.md) - Managing Conflicts |
| ...troubleshoot deployment? | [Troubleshooting Guide](operations/troubleshooting.md) |
| ...use the API? | [API Reference](developer/api-reference.md) |
| ...contribute code? | [Development Guide](contributing/Development.md) |
| ...contribute code? | [Development Guide](contributing/development.md) |
### "Where is..."
@@ -112,7 +112,7 @@ Complete guide to Bookhoard documentation. Find what you need quickly.
### Set up a new device
1. Choose your device: [Kobo](user/devices/kobo-setup.md) or [KOReader](user/devices/koreader-setup.md)
1. Choose your device: [KOReader](user/devices/koreader-setup.md) (works on Kindle, Kobo, and PocketBook hardware)
2. Understand sync: [Sync Guide](user/sync-guide.md)
3. Troubleshoot: Device-specific guides
@@ -128,7 +128,7 @@ Complete guide to Bookhoard documentation. Find what you need quickly.
1. Follow [README.md](../README.md) quick start
2. Configure environment: [.env.example](../.env.example)
3. Review [Troubleshooting Guide](operations/troubleshooting.md)
4. Check [Development Guide](contributing/Development.md) for performance tuning
4. Check [Development Guide](contributing/development.md) for performance tuning
---
@@ -138,12 +138,12 @@ When adding new features:
1. **User-facing features** → Update relevant User docs
2. **API endpoints** → Update [API Reference](developer/api-reference.md) & split docs
3. **Backend changes** → Update [Development Guide](contributing/Development.md)
3. **Backend changes** → Update [Development Guide](contributing/development.md)
4. **Deployment changes** → Update [Operations Portal](operations/operations.md)
Keep [PROJECT_GUIDELINES.md](PROJECT_GUIDELINES.md) in mind for documentation standards.
Keep [../PROJECT_GUIDELINES.md](../PROJECT_GUIDELINES.md) in mind for documentation standards.
---
**Last Updated**: 2026-02-08
**Last Updated**: August 2026
**Bookhoard Version**: 1.0
+5 -5
View File
@@ -8,12 +8,12 @@ When creating or managing a library, you can add folders containing your media f
The admin library page includes a folder browser to help you select folders on the server:
1. Navigate to **Admin → Library Management**
2. Find the library you want to manage
3. Click the **Folders** button
4. Click **Browse** next to "Add folder path"
1. Open the **Administration** panel in the sidebar (admins only) and go to **Libraries**
2. Click a library in the list to expand its panel
3. Find the **Folders** section
4. Click **Browse** next to the folder path input — this opens the **Browse Folders** dialog
5. Navigate through the server's filesystem
6. Select a folder by clicking **Select This Folder**
6. Select a folder; it fills the path input, then click **Add**
### Security
+20 -20
View File
@@ -58,20 +58,22 @@ Calibre Library/
### Step 2: Add Library in Bookhoard
1. Navigate to **Admin** **Libraries**
2. Click **Add Library**
1. Open the **Administration** panel in the sidebar and go to **Libraries**
2. Click **Create Library**
3. Configure:
- **Name**: "My Calibre Library"
- **Type**: Ebook (or Audiobook/Comic)
- **Folder**: Path to your Calibre library
- **Scan on save**: ✅ Checked
4. Click **Save**
- **Library Name**: "My Calibre Library"
- **Description**: Optional
- **Library Type**: Ebook (or Audiobook/Comic)
4. Click the library in the list to expand its panel
5. Add your Calibre library folder in the **Folders** section:
- Enter the path (or click **Browse** to find it on the server) and click **Add**
6. Trigger a scan (see below), or rely on watch mode if enabled
Bookhoard will automatically scan the library and import all books with their Calibre metadata.
Bookhoard scans the library and imports all books with their Calibre metadata. Scan progress shows in the sidebar next to the logo.
### Step 3: Verify Import
1. Navigate to **Library** view
1. Open the **Dashboard** or **All Books** page (sidebar navigation)
2. Browse your imported books
3. Check that:
- Titles and authors are correct
@@ -136,8 +138,8 @@ As long as a `metadata.opf` file exists in the folder, Bookhoard will import the
**Scenario**: You have a Calibre library with 500 ebooks, all organized with series, tags, and custom covers.
**Steps**:
1. Add the Calibre library folder in Bookhoard
2. Enable "Scan on save"
1. Add the Calibre library folder in Bookhoard (Administration → Libraries → expand the library → **Folders**)
2. Trigger a scan via the **Scanner API**, or let watch mode pick up the changed files (the File Watcher status is shown on the admin dashboard)
3. Bookhoard imports all 500 books with:
- Correct titles and authors
- Series information (e.g., "Harry Potter #2")
@@ -166,9 +168,8 @@ As long as a `metadata.opf` file exists in the folder, Bookhoard will import the
**Steps**:
1. Edit metadata in Calibre (it updates `metadata.opf`)
2. In Bookhoard, trigger a rescan:
- Navigate to **Admin****Libraries**
- Click **Rescan** on your library
- Or use the **Scanner API** to force rescan
- Via the **Scanner API** (`POST /api/scanner/scan`), or
- Let watch mode detect the changed files automatically (see File Watcher on the admin dashboard)
3. Bookhoard detects updated `metadata.opf` and refreshes metadata
**Result**: Bookhoard reflects your Calibre changes automatically.
@@ -182,7 +183,7 @@ As long as a `metadata.opf` file exists in the folder, Bookhoard will import the
**Solutions**:
1. **Check file structure**: Ensure `metadata.opf` is in the same folder as the book file
2. **Verify library type**: Ensure library type matches content (ebook vs. audiobook)
3. **Force rescan**: Use the "Force Rescan" option to re-import all metadata
3. **Force rescan**: Trigger a scan via the Scanner API to re-import all metadata (watch mode also picks up changed files automatically)
4. **Check logs**: Review Bookhoard logs for parsing errors
### Incorrect Metadata
@@ -219,7 +220,7 @@ As long as a `metadata.opf` file exists in the folder, Bookhoard will import the
**Do**:
- ✅ Edit metadata in Calibre
-Rescan in Bookhoard to sync changes
-Let Bookhoard's next scan (or watch mode) pick up the changes
- ✅ Use Calibre for library management
**Don't**:
@@ -259,12 +260,11 @@ Stay tuned for updates!
### OPDS Integration
You can access your Bookhoard library (including Calibre-imported books) via OPDS from Calibre-aware devices:
- Kobo e-readers
- KOReader
You can access your Bookhoard library (including Calibre-imported books) via OPDS from OPDS-capable clients:
- KOReader (Kindle, Kobo, PocketBook hardware)
- Phone/tablet apps (KYBook, Chunky, etc.)
See the [Kobo Setup Guide](devices/kobo-setup.md) or [KOReader Setup Guide](devices/koreader-setup.md) for details.
See the [KOReader Setup Guide](devices/koreader-setup.md) for details.
## FAQ
+19 -3
View File
@@ -6,7 +6,7 @@ Collections allow you to organize books across multiple libraries.
### From Collections Page
Navigate to `/collections` to see all your collections. Clicking on a collection shows ALL books in that collection across all libraries.
Navigate to **Collections** (sidebar navigation) to see all your collections. Clicking on a collection shows ALL books in that collection across all libraries.
### From Dashboard
@@ -19,8 +19,24 @@ When viewing a specific library's dashboard, collections only show books from th
## Creating Collections
[Instructions for creating collections]
1. Go to **Collections** (sidebar navigation)
2. Click **New Collection** (top-right) — or **Create Your First Collection** if the list is empty
3. Fill in the details:
- **Name** - Collection name
- **Description** - Optional description
- **Icon** - Pick from the icon grid
- **Color** - Pick a color swatch
4. Click **Create Collection**
## Managing Collections
[Instructions for editing/deleting collections]
Each collection in the list has icon buttons on its card:
- **Edit** - Opens the edit form to change name, description, icon, or color. Click **Update Collection** to save.
- **Delete** - Removes the collection after a confirmation prompt.
Deleted a system collection by mistake? The **Restore System** button on the Collections page brings back system collections.
### Dashboard Sections
Collections appear as sections on your dashboard. Show, hide, and reorder them from the dashboard's **Customize Dashboard** settings (see [Dashboard](dashboard.md)).
+14 -13
View File
@@ -16,28 +16,27 @@ Smart sections are automatically generated based on your reading activity:
### User Collections
Any collection marked with "Show on Dashboard" will appear as a section on your dashboard.
Your collections appear as sections on the dashboard. To show or hide a collection's section:
To enable a collection:
1. Go to Collections
2. Edit a collection
3. Toggle "Show on Dashboard"
4. Save
1. Select the library in the **Library** bar
2. Click the **Customize Dashboard** icon button
3. Toggle the collection on or off
4. Click "Save Changes"
### Customizing Your Dashboard
1. Click the ⚙️ (gear icon) in the top-right
2. **Drag sections** to reorder them
3. **Toggle visibility** with the switches
4. **Adjust items per section** (10-50 items)
5. Click "Save Changes"
1. In the **Library** bar below the top bar, select the library you want to customize (a specific library, not "All Libraries")
2. Click the **Customize Dashboard** icon button at the right end of the Library bar (next to Refresh)
3. **Drag sections** to reorder them
4. **Toggle visibility** with the switches
5. **Adjust items per section** (10-50 items)
6. Click "Save Changes"
Settings are saved per library.
### Library Switching
Use the dropdown in the sticky header to switch between libraries. Each library has its own dashboard settings.
Use the **Library** dropdown in the bar below the top bar to switch between libraries (including "All Libraries"). Each library has its own dashboard settings.
### Keyboard Navigation
@@ -45,6 +44,8 @@ Use the dropdown in the sticky header to switch between libraries. Each library
- **Arrow Keys**: Scroll carousels horizontally
- **Enter**: Open selected book
Hovering a carousel shows chevron buttons on either side for scrolling.
### Touch Gestures (Mobile)
- **Swipe**: Drag carousel left/right to scroll
+24 -632
View File
@@ -1,650 +1,42 @@
# Kobo Device Setup Guide
This guide will help you set up your Kobo e-reader to sync with Bookhoard for seamless cross-device reading progress synchronization.
> ## 🚧 Coming Soon
>
> Native Kobo sync is not available yet. It is actively being developed and this guide will be filled in as the feature lands.
## What is Kobo Sync?
## Using a Kobo With Bookhoard Today
Bookhoard implements a Kobo-compatible sync protocol that allows your Kobo device to:
You don't have to wait: **KOReader runs on Kobo hardware** and syncs fully with Bookhoard today — reading position, bookmarks, highlights, and notes, plus OPDS wireless book delivery.
- Sync reading progress across all your devices
- Sync highlights and bookmarks
- Sync reading statistics
- Maintain device-specific metadata
See the **[KOReader Setup Guide](koreader-setup.md)** for complete instructions.
## Prerequisites
## What's Planned for Native Kobo Sync
Before you begin, make sure you have:
When released, native Kobo sync will let stock Kobo firmware talk directly to Bookhoard:
- ✅ A Kobo e-reader device (Clara, Aura, Nia, Libra, Sage, Elipsa, etc.)
- ✅ A Bookhoard instance running and accessible on your network
- ✅ Your Bookhoard credentials (username and password)
- ✅ USB cable to connect your Kobo to your computer
- ✅ Your Kobo connected to the same Wi-Fi network as your Bookhoard instance
- **Reading position sync** — percentages, pages, and reading statistics
- **Bookmarks, highlights, and notes** — synced with the web and other devices
- **OPDS wireless delivery** — browse and download books directly on the Kobo
- **Automatic EPUB → KEPUB conversion** — for better Kobo rendering
- **Shelf mappings** — Bookhoard collections appearing as Kobo shelves
## Supported Kobo Devices
Bookhoard supports all Kobo devices that use the standard Kobo sync protocol:
- **Kobo Clara**: Clara 2E, Clara HD
- **Kobo Aura**: Aura, Aura H2O, Aura ONE, Aura Edition 2
- **Kobo Libra**: Libra 2, Libra H2O
- **Kobo Forma**: All versions
- **Kobo Sage**: All versions
- **Kobo Elipsa**: All versions
- **Kobo Nia**: All versions
- **Kobo Touch**: Touch 2.0
- **Kobo Glo**: Glo, Glo HD
## Device Registration
### Step 1: Find Your Kobo Serial Number
1. Turn on your Kobo device
2. Go to **Settings** (gear icon)
3. Select **Device Information**
4. Note your **Device Serial Number** (e.g., N1234567890123)
- This is your device identifier for registration
### Step 2: Register Your Device in Bookhoard
1. Log in to your Bookhoard web interface
2. Navigate to **Device Management****Add New Device**
3. Fill in the device details:
- **Device Name**: A friendly name (e.g., "My Kobo Clara")
- **Device Type**: Select "Kobo"
- **Device Identifier**: Enter your Kobo serial number
4. Click **Register Device**
You'll receive:
- An **Auth URL** to approve the device
- Instructions for manual configuration
### Step 3: Approve Your Device
1. **Method A: QR Code**
- If displayed, scan the QR code with your phone's camera
- This will open the approval page in your browser
- Log in and click **Approve**
2. **Method B: Manual URL**
- Copy the Auth URL from the registration confirmation
- Open it in your web browser
- Log in to your Bookhoard account
- Click **Approve Device**
Your device is now registered and ready for configuration!
## Configure Kobo Sync
### Step 1: Connect Kobo to Your Computer
1. Use your USB cable to connect Kobo to your computer
2. Your computer should recognize Kobo as a storage device
3. Kobo will show "Connected" and "Eject before disconnecting"
### Step 2: Edit Kobo Configuration File
#### Windows Users
1. Open **File Explorer** and navigate to your Kobo device
2. Open the `.kobo` folder (hidden folder)
3. Open `Kobo/Kobo eReader.conf` in a text editor (Notepad++, VS Code, etc.)
#### Mac Users
1. Kobo device appears on your Desktop
2. Right-click the Kobo volume and select **Show Package Contents**
3. Navigate to `.kobo/Kobo/Kobo eReader.conf`
4. Open in a text editor (TextEdit, VS Code, etc.)
#### Linux Users
1. Kobo mounts at `/media/USERNAME/Kobo` or similar
2. Navigate to `.kobo/Kobo/Kobo eReader.conf`
3. Open in a text editor
### Step 3: Add Bookhoard Sync Configuration
After device registration is complete, you'll receive an API key and sync URL from Bookhoard.
Add the following section to the end of your `Kobo eReader.conf` file:
```ini
[FeatureSettings]
# Enable Kobo store replacement
KoboStoreSyncDisabled=true
[Sync]
# Bookhoard Sync Configuration (from Device Management page)
ServerURL=http://YOUR_COMPUTER_IP:8765/api/sync/kobo/YOUR_API_KEY
AutoSyncEnabled=true
SyncFrequency=5
```
**Where to find these values**:
- `YOUR_COMPUTER_IP`: Your Bookhoard server's IP address (e.g., 192.168.1.100)
- `YOUR_API_KEY`: Copy from Bookhoard Device Management → Your Kobo Device → "Copy Sync URL"
**Example configuration**:
```ini
[Sync]
ServerURL=http://192.168.1.100:8765/api/sync/kobo/dev_abc123def456
AutoSyncEnabled=true
SyncFrequency=5
```
**Important Notes**:
- The API key is generated during device registration
- You can regenerate the API key anytime from Device Management if needed
- Keep your API key confidential like a password
- Bookhoard uses revocable API keys for security (not username/password)
**Replace the following with your actual values**:
- `YOUR_COMPUTER_IP`: Your computer's local IP address (e.g., 192.168.1.100)
- `YOUR_BOOKHOARD_USERNAME`: Your Bookhoard email or username
- `YOUR_BOOKHOARD_PASSWORD`: Your Bookhoard password
**Example configuration:**
```ini
[Sync]
ServerURL=http://192.168.1.100:8765/api/sync/kobo
AutoSyncEnabled=true
SyncFrequency=5
Username=john@example.com
Password=securePassword123
```
### Step 4: Save and Eject
1. Save the `Kobo eReader.conf` file
2. Safely eject your Kobo device from your computer
3. Kobo will restart automatically
### Step 5: Verify Sync on Kobo
1. After Kobo restarts, go to **Settings****Sync & Backup**
2. You should see "Bookhoard" listed as a sync provider
3. Tap **Sync Now** to test the connection
4. If successful, you'll see a "Sync Complete" message
## Sync Features
### Reading Progress Sync
Kobo syncs:
- **Percentage Read**: Overall book completion percentage
- **Page Number**: Current page in fixed-layout books
- **Time Spent**: Reading time statistics
- **Last Read**: Timestamp of last reading session
### Annotations Sync
Kobo syncs:
- **Bookmarks**: Page positions saved for quick access
- **Highlights**: Highlighted text passages
- **Notes**: Notes attached to highlights
- **Reading Statistics**: Pages read, time spent
### Shelf Management
Kobo syncs:
- **Book Collections**: Your organized shelves
- **Shelf Contents**: Books in each collection
- **Sync Metadata**: When shelves were last updated
## OPDS Wireless Book Delivery
### What is OPDS?
OPDS (Open Publication Distribution System) allows your Kobo to **wirelessly download books** from Bookhoard - no USB cable needed!
### OPDS Benefits
- **No USB Required**: Download books directly to your Kobo over Wi-Fi
- **On-Demand Delivery**: Browse your Bookhoard library from your Kobo
- **Collection Support**: Download books from specific collections
- **Progress Tracking**: Books downloaded via OPDS sync progress automatically
- **Format Conversion**: Automatic EPUB to KEPUB conversion for better Kobo support
### Enable OPDS on Your Kobo
#### Option 1: Automatic Configuration (Recommended)
1. After registering your Kobo device, a **Download Configuration** button appears
2. Click **Download Configuration** to get a `.kobo` configuration file
3. Copy this file to your Kobo's `.kobo/` directory via USB
4. Eject and restart your Kobo
5. OPDS catalog will automatically appear in your Kobo's store
#### Option 2: Manual Configuration
1. Connect your Kobo to your computer via USB
2. Navigate to `.kobo/Kobo/Kobo eReader.conf`
3. Add the following configuration:
```ini
[FeatureSettings]
# Enable OPDS catalog
OPDSCatalogEnabled=true
OPDSCatalogURL=http://YOUR_COMPUTER_IP:8765/opds/devices/YOUR_DEVICE_ID/catalog?token=YOUR_API_KEY
# Example:
# OPDSCatalogURL=http://192.168.1.100:8765/opds/devices/kobo-clara-123/catalog?token=dev_abc123def456
```
4. Replace:
- `YOUR_COMPUTER_IP`: Your Bookhoard server IP
- `YOUR_DEVICE_ID`: Your Kobo's device ID from Bookhoard Device Management
- `YOUR_API_KEY`: Your Kobo device's API key (same as in sync URL)
5. Save the file and safely eject your Kobo
### Access OPDS Catalog on Kobo
1. Wake your Kobo and connect to Wi-Fi
2. Go to **Home****Store** (or **Shop**)
3. You'll see **Bookhoard** listed as a store
4. Tap to enter the Bookhoard catalog
### Browse and Download Books
#### Browse All Books
1. In the Bookhoard catalog, you'll see all books from your library
2. Browse by:
- **Recently Added**: Latest books in your library
- **Collections**: Books organized by collections
- **Authors**: Books grouped by author
- **Series**: Books in reading order
#### Download a Book
1. Tap on any book cover to see details
2. Tap **Download** or **Add to Library**
3. The book downloads wirelessly to your Kobo
4. Progress bar shows download status
5. Once downloaded, the book appears in your **Home** library
#### Download from Collections
1. In the Bookhoard catalog, tap **Collections**
2. Select a collection (e.g., "Science Fiction")
3. Browse books in that collection
4. Tap to download individual books
5. Or tap **Download All** to get entire collection
### OPDS Features
#### Format Support
Kobo OPDS supports:
- **EPUB**: Standard ebook format (recommended)
- **KEPUB**: Kobo-optimized EPUB (better page turns, fonts)
- **PDF**: Fixed-layout documents
**Automatic Conversion**: Bookhoard automatically converts EPUB to KEPUB on-the-fly for better Kobo experience.
#### Progress Sync
Books downloaded via OPDS automatically sync progress:
1. Download a book via OPDS
2. Start reading on your Kobo
3. Progress syncs to Bookhoard automatically
4. Continue reading on any other device!
#### Collection to Shelf Mapping
Bookhoard maps your collections to Kobo shelves:
- Collection **"Science Fiction"** → Kobo shelf **"Sci-Fi"**
- Collection **"To Read"** → Kobo shelf **"To Read"**
- Customizable in Bookhoard Device Management
### OPDS Troubleshooting
#### Catalog Not Appearing
**Problem**: Bookhoard catalog doesn't show in Kobo store
**Solutions**:
1. Verify OPDS URL is correct in config file
2. Check Kobo is connected to Wi-Fi
3. Try accessing OPDS URL in your browser
4. Ensure device ID matches Bookhoard device ID
5. Restart Kobo after editing config file
#### Download Fails
**Problem**: Book download starts but fails partway through
**Solutions**:
1. Check Wi-Fi signal strength
2. Ensure Bookhoard server is running
3. Verify book file exists in Bookhoard library
4. Try downloading a smaller book first
5. Check Bookhoard logs for errors
#### Book Downloads But Won't Open
**Problem**: Downloaded book shows error when opening
**Solutions**:
1. Verify book format is supported (EPUB/KEPUB/PDF)
2. Check file isn't corrupted in Bookhoard
3. Try downloading via USB and opening
4. Check Kobo has sufficient free storage
5. Restart your Kobo device
#### Slow Download Speed
**Problem**: Books take too long to download
**Solutions**:
1. Ensure strong Wi-Fi signal (stay near router)
2. Use 5GHz Wi-Fi if your Kobo supports it
3. Close other apps using bandwidth
4. Download smaller books first
5. Consider using USB for large books
### OPDS vs USB Transfer
| Feature | OPDS (Wireless) | USB Transfer |
| -------------------- | ----------------------------- | ------------------------- |
| **Convenience** | ⭐⭐⭐⭐⭐ No cable needed | ⭐⭐ Requires cable |
| **Speed** | ⭐⭐⭐ Fast (Wi-Fi dependent) | ⭐⭐⭐⭐⭐ Very fast |
| **Bulk Transfer** | ⭐⭐⭐ One at a time | ⭐⭐⭐⭐⭐ Many at once |
| **Progress Sync** | ⭐⭐⭐⭐⭐ Automatic | ⭐⭐⭐⭐ After first sync |
| **Setup Complexity** | ⭐⭐⭐ Moderate | ⭐⭐⭐⭐⭐ Simple |
| **Reliability** | ⭐⭐⭐⭐ Good | ⭐⭐⭐⭐⭐ Excellent |
**Recommendation**: Use OPDS for convenience (1-5 books), use USB for bulk transfers (10+ books).
### Advanced OPDS Configuration
#### Custom Catalog Name
Change the name of the Bookhoard catalog on your Kobo:
```ini
[OPDS]
CatalogName=My Library
```
#### Auto-Download
Automatically download new books added to collections:
```ini
[OPDS]
AutoDownloadEnabled=true
AutoDownloadCollections=To Read,Recent
```
#### Download Quality
Choose between original EPUB or converted KEPUB:
```ini
[OPDS]
PreferredFormat=kepub # Options: epub, kepub, auto
```
## Sync Frequency Options
Configure how often Kobo syncs with Bookhoard:
```ini
[Sync]
# Sync frequency in minutes
SyncFrequency=5 # Sync every 5 minutes (recommended)
SyncFrequency=15 # Sync every 15 minutes
SyncFrequency=60 # Sync every hour
SyncFrequency=0 # Manual sync only
```
**Recommended**: `SyncFrequency=5` for near real-time sync
**Battery Saving**: `SyncFrequency=15` or `30` to reduce Wi-Fi usage
**Manual Only**: `SyncFrequency=0` sync only when you press "Sync Now"
## Manual Sync
To manually trigger a sync on your Kobo:
1. Connect Kobo to Wi-Fi
2. Go to **Settings****Sync & Backup**
3. Tap **Sync Now**
4. Wait for "Sync Complete" message
## Advanced Configuration
### Disable Kobo Store
To prevent Kobo from trying to connect to the official Kobo store:
```ini
[FeatureSettings]
KoboStoreSyncDisabled=true
```
### Custom Sync URL
If you're running Bookhoard with a custom domain or port:
```ini
[Sync]
# Custom domain
ServerURL=https://bookhoard.example.com/api/sync/kobo
# Custom port
ServerURL=http://192.168.1.100:9000/api/sync/kobo
# Localhost (for testing)
ServerURL=http://localhost:8765/api/sync/kobo
```
### HTTPS Configuration
If you have SSL/TLS configured on Bookhoard:
```ini
[Sync]
ServerURL=https://bookhoard.yourdomain.com/api/sync/kobo/YOUR_API_KEY
```
Replace `YOUR_API_KEY` with your device's API key from Bookhoard Device Management.
Kobo will automatically trust the certificate if properly configured.
## Troubleshooting
### Sync Not Working
**Problem**: Sync doesn't happen automatically
**Solutions**:
1. Check Kobo is connected to Wi-Fi
2. Verify `AutoSyncEnabled=true` in config
3. Check `SyncFrequency` is not set to 0
4. Test with manual sync first
5. Check Bookhoard logs for connection attempts
### Connection Refused
**Problem**: "Connection refused" or "Server not reachable"
**Solutions**:
1. Verify Bookhoard is running on your computer
2. Check the server URL and IP address are correct
3. Ensure Kobo is on same Wi-Fi network as computer
4. Temporarily disable firewall to test
5. Try accessing Bookhoard URL in your browser first
### Authentication Failed
**Problem**: "Authentication failed" or "Invalid API key"
**Solutions**:
1. Verify the API key in your sync URL matches the one in Bookhoard Device Management
2. Check that device is approved in Bookhoard (not pending)
3. Try regenerating the API key from Device Management page
4. Ensure the sync URL is complete (includes the API key)
5. Copy the sync URL directly from Device Management → "Copy Sync URL" button
### Configuration File Not Saving
**Problem**: Changes to `Kobo eReader.conf` are lost
**Solutions**:
1. Make sure Kobo is ejected safely after editing
2. Check file permissions (should be writable)
3. Try a different text editor (Notepad++, VS Code, Sublime Text)
4. Backup the file before editing
5. On Mac, ensure you're not editing the package directly
### Sync Only Works Manually
**Problem**: Manual sync works, but auto-sync doesn't
**Solutions**:
1. Verify `AutoSyncEnabled=true` in config
2. Check `SyncFrequency` is not 0
3. Kobo only syncs when connected to Wi-Fi
4. Some Kobo models require Wi-Fi to be manually connected
5. Check Bookhoard device management page for connection errors
### Books Not Appearing in Kobo
**Problem**: Books added to Bookhoard don't show on Kobo
**Solutions**:
1. Kobo needs books to be sideloaded (manually transferred via USB)
2. Bookhoard syncs PROGRESS, not book files
3. Transfer book files to Kobo's `Documents` folder via USB
4. Kobo will then sync progress for those books with Bookhoard
5. Check that book formats are supported by Kobo
### Conflicts Not Showing
**Problem**: Conflicts between devices aren't being detected
**Solutions**:
1. Check Bookhoard Conflicts page
2. Ensure both devices have synced recently
3. Conflicts only detected when progress differs within 5 minutes
4. Manually sync both devices to trigger conflict detection
5. Review conflict resolution settings
## Security Best Practices
1. **Use HTTPS**: If deploying Bookhoard publicly, configure SSL/TLS
2. **Strong Password**: Use a secure password for your Bookhoard account
3. **Network Security**: Ensure your Wi-Fi network is secure (WPA2/WPA3)
4. **Regular Updates**: Keep Kobo firmware updated
5. **Device Authorization**: Only approve devices you recognize
## Network Configuration
### Local Network (Recommended)
For home use, keep Kobo and Bookhoard on the same local network:
```
Kobo Wi-Fi: 192.168.1.x
Bookhoard: 192.168.1.x
```
### Remote Access
For access outside your home network:
1. Set up port forwarding on your router (port 8765)
2. Configure SSL/TLS on Bookhoard
3. Use a dynamic DNS service for constant hostname
4. Update Kobo config with public URL including API key:
```ini
[Sync]
ServerURL=https://yourdomain.com/api/sync/kobo/YOUR_API_KEY
```
## Performance Optimization
### Battery Life
To extend Kobo battery life:
1. Use longer sync intervals (15-30 minutes)
2. Sync only on Wi-Fi (not cellular if your Kobo has it)
3. Disable unnecessary Kobo features
4. Keep Kobo in sleep mode when not reading
### Sync Speed
To improve sync speed:
1. Ensure strong Wi-Fi signal
2. Use local network (not remote access)
3. Keep Bookhoard and Kobo on same network
4. Close other apps using Wi-Fi bandwidth
5. Reduce number of books syncing at once
## Additional Resources
- [Kobo Developer Documentation](https://help.kobo.com/hc/en-us)
- [Bookhoard Universal Sync Guide](../sync-guide.md)
- [KOReader Setup Guide](koreader-setup.md)
- [Bookhoard API Reference](../../developer/api-reference.md)
The server-side protocol endpoints are already implemented and under test; the feature will be announced when it's ready for real devices.
## FAQ
**Q: Can I sync books (files) between devices?**
A: No, Bookhoard only syncs reading progress and annotations. You must sideload book files to each device manually.
**Q: Should I buy a Kobo to use with Bookhoard today?**
A: Kobo devices work great with Bookhoard via KOReader. Native (stock firmware) sync is coming soon.
**Q: Will Kobo update automatically when I add books in Bookhoard?**
A: No, Kobo doesn't fetch book files from Bookhoard. You must transfer books via USB.
**Q: What happens to my KOReader setup when native sync arrives?**
A: Nothing — KOReader will keep working. Native sync simply adds another option for people who prefer stock Kobo firmware.
**Q: Can I use both Kobo Sync and Calibre?**
A: Yes, but they may conflict. It's recommended to choose one sync method.
## Additional Resources
**Q: What happens if I read the same book on Kobo and KOReader?**
A: Bookhoard will detect conflicts and you can resolve them in the Conflicts UI.
**Q: Does Kobo sync when in sleep mode?**
A: Only if Wi-Fi is enabled and configured to stay active during sleep.
## Support
If you encounter issues:
1. Check the troubleshooting section above
2. Review Kobo sync logs in device settings
3. Check Bookhoard sync queue and device management pages
4. Verify your configuration file is saved correctly
5. Open an issue on the Bookhoard GitHub repository
- [KOReader Setup Guide](koreader-setup.md) — works on Kobo today
- [Bookhoard Universal Sync Guide](../sync-guide.md)
- [Bookhoard API Reference](../../developer/api-reference.md)
---
**Last Updated**: 2026-01-31
**Bookhoard Version**: 1.0
**Kobo Firmware**: 4.30.0+
**Last Updated**: August 2026
**Bookhoard Version**: 1.0
+70 -406
View File
@@ -9,18 +9,19 @@ KOReader is an open-source e-reader application that supports a wide range of e-
- Kindle devices (Paperwhite, Oasis, Voyage, etc.)
- Kobo devices (Clara, Aura, Nia, etc.)
- PocketBook devices
- Android tablets and phones
It also runs on Android tablets and phones, although Bookhoard's dedicated mobile apps (coming later) will be the better option there.
## Prerequisites
Before you begin, make sure you have:
- ✅ A Bookhoard instance running and accessible on your network
-Your Bookhoard credentials (username and password)
-A web browser logged in to your Bookhoard account (for device approval)
- ✅ A KOReader-compatible e-reader device
- ✅ Your device connected to the same Wi-Fi network as your Bookhoard instance
## Installation
## Installing KOReader
### Kindle Devices
@@ -64,469 +65,132 @@ Before you begin, make sure you have:
- Open KOReader from your apps menu
- Enable Wi-Fi in the network settings
## Device Registration
## Connecting KOReader to Bookhoard
### Step 1: Get Your Bookhoard Instance URL
Setup is done **on the server**: you approve the device from the Bookhoard web interface — no usernames, passwords, or tokens to type on the device.
Find your Bookhoard instance URL. This will typically be one of:
### Step 1: Install the Bookhoard Plugin
- **Local Network**: `http://YOUR_COMPUTER_IP:8765`
- **Localhost (if testing)**: `http://localhost:8765`
- **Domain (if configured)**: `https://bookhoard.yourdomain.com`
1. Clone the [Bookhoard KOReader plugin](https://git.linuxhg.com/Bookhoard/bookhoard.koplugin)
2. Copy it to your KOReader `plugins/` directory
3. Restart KOReader
### Step 2: Register Your Device in Bookhoard
### Step 2: Point the Plugin at Your Server
1. Log in to your Bookhoard web interface
2. Navigate to **Device Management****Add New Device**
3. Fill in the device details:
- **Device Name**: A friendly name (e.g., "My Kindle Paperwhite")
- **Device Type**: Select "KOReader"
- **Device Identifier**: Enter your device's hardware ID or serial number
- On Kindle: Settings → Device Options → Device Info → Serial Number
- On Kobo: Settings → Device Information → Serial Number
4. Click **Register Device**
You'll receive:
- An **Auth URL** to approve the device
- A **Device Token** (automatically generated after approval)
### Step 3: Approve Your Device
1. **Method A: QR Code**
- If displayed, scan the QR code with your phone's camera
- This will open the approval page in your browser
- Log in and click **Approve**
2. **Method B: Manual URL**
- Copy the Auth URL from the registration confirmation
- Open it in your web browser
- Log in to your Bookhoard account
- Click **Approve Device**
Your device is now registered and ready to sync!
## Configure KOReader Sync
### Step 1: Access KOReader Settings
1. Open KOReader on your device
2. Tap the menu icon (≡) in the top-left corner
3. Select **Tools****Calibre**
### Step 2: Configure Wireless Connection
1. **Enable Calibre Wireless Connection**: Toggle ON
2. **Server Address**: Enter your Bookhoard instance URL
1. Open KOReader, tap the **wrench icon** at the top
2. Find and tap **Bookhoard sync**
3. Tap **Server URL**, enter your server address, then tap **OK**:
```
http://YOUR_COMPUTER_IP:8765/api/sync/koreader
http://YOUR_COMPUTER_IP:8765
```
Replace `YOUR_COMPUTER_IP` with your actual IP address
Use your server's LAN IP (or domain if you have one configured).
3. **Set Custom Port** (if needed): Keep default or enter `8765`
### Step 3: Approve the Device in Bookhoard
### Step 3: Configure Authentication
1. On your computer or phone, open Bookhoard and go to the **Devices** page (sidebar navigation)
2. Refresh the page — your device appears under **Pending Device Registrations**
3. Click **Approve** to connect the device
1. **Authentication Method**: Select "Basic Auth"
2. **Username**: Your Bookhoard email or username
3. **Password**: Your Bookhoard password
Once approved, the plugin picks up its credentials automatically — reading progress sync and OPDS catalog access are set up automatically. No further configuration is needed.
### Step 4: Configure Sync Settings
> **Note:** Pending registrations expire after 5 minutes. If yours expires, just re-run the sync from the plugin menu and approve again.
1. **Auto Sync**: Enable for automatic sync
2. **Sync Frequency**: Choose from:
- Every page turn (recommended for real-time sync)
- Every bookmark save
- Every highlight
- Manual only (sync when you press the sync button)
### Auth Token (Advanced)
3. **What to Sync**: Enable:
- ✅ Reading progress
- ✅ Bookmarks
- ✅ Highlights
- ✅ Notes
The Devices page shows each KOReader device's **Auth Token**. You normally never need it (the plugin receives it automatically during approval), but it can be re-entered manually in the plugin settings if you're moving a setup between devices or debugging.
### Step 5: Test Connection
## What Syncs
1. Tap **Test Connection** in the Calibre settings
2. You should see a success message if configured correctly
3. If it fails:
- Verify your device is connected to Wi-Fi
- Check the server URL is correct
- Ensure your Bookhoard instance is running
- Verify username and password are correct
Once connected, the following sync automatically in both directions between KOReader and Bookhoard (web and other devices):
## Using Sync Features
- **Reading position** — percentage, chapter, and EPUB CFI where available
- **Bookmarks**
- **Highlights** — including highlight colors, mapped between the web and KOReader palettes
- **Notes** — standalone and attached to highlights
### Initial Sync
When you first enable sync, KOReader will:
1. Connect to Bookhoard
2. Upload your current reading progress
3. Download any annotations from the server
4. Set up bidirectional sync for future changes
### Reading Progress Sync
As you read:
- Progress updates automatically sync based on your sync frequency
- Page turns, chapter changes, and bookmark saves all trigger sync
- Sync occurs in the background without interrupting reading
### Annotations Sync
- **Bookmarks**: Sync when created or deleted
- **Highlights**: Sync when created, edited, or deleted
- **Notes**: Sync when created, edited, or deleted
- **Linked Notes**: Notes attached to highlights sync together
### Manual Sync
To manually trigger a sync:
1. Open the KOReader menu (≡)
2. Select **Tools** → **Calibre**
3. Tap **Sync Now**
The sync status will display:
- 🟢 **Synced** - All changes uploaded
- 🟡 **Syncing...** - In progress
- 🔴 **Failed** - Check your network connection
## Advanced Configuration
### Offline Mode
KOReader automatically handles offline scenarios:
1. Changes are queued locally when offline
2. Auto-sync resumes when connected
3. Queue processes all pending changes in priority order
### Checkpoint Sync
For better battery life, use checkpoint mode:
1. In KOReader Calibre settings
2. Set **Sync Mode** to "Checkpoint"
3. Set **Checkpoint Interval** (e.g., every 5 minutes)
4. Syncs occur in batches instead of every action
### Debug Mode
Enable debug logging if sync isn't working:
1. KOReader menu → Tools → Calibre
2. Enable **Debug Logging**
3. Sync and check logs at `/mnt/us/koreader/calibre.log`
Books are matched automatically using UUIDs, file hashes (SHA-256, format-aware so converted files still match), file aliases, and title/author fallback. If a book can't be matched, it shows up under the device's **Unlinked Books** in Bookhoard, where you can link it manually.
## OPDS Wireless Book Delivery
### What is OPDS?
OPDS (Open Publication Distribution System) allows your KOReader device to **wirelessly download books** from Bookhoard - no USB cable needed!
### OPDS Benefits
- **Wireless Downloads**: Browse and download books over Wi-Fi
- **On-Demand Access**: Your entire library at your fingertips
- **Collection Support**: Browse and download from specific collections
- **Automatic Progress Sync**: Downloaded books sync progress instantly
- **Format Support**: EPUB, KEPUB, PDF, and more
### Enable OPDS in KOReader
#### Step 1: Get Your OPDS URL
1. Log in to Bookhoard web interface
2. Go to **Device Management**
3. Find your registered KOReader device
4. Click **Show OPDS URL**
5. Copy the URL (format: `http://YOUR_IP:8765/opds/devices/YOUR_DEVICE_ID/catalog`)
#### Step 2: Add OPDS Catalog in KOReader
1. Open KOReader on your device
2. Tap the **+** (plus) button on the home screen
3. Select **OPDS Catalog**
4. Enter catalog details:
- **Name**: Bookhoard (or any name you prefer)
- **URL**: Paste your OPDS URL from Step 1
5. Tap **Save**
Your Bookhoard library now appears in KOReader's home screen!
Once your device is approved, the plugin also registers Bookhoard's OPDS catalog, so you can browse and download books wirelessly — no USB cable needed.
### Browse and Download Books
#### Browse Your Library
1. In KOReader, open the OPDS catalog list and tap **Bookhoard**
2. Browse your library: all books, collections, and recent additions
3. Tap a book to see details and **Download** it
1. Tap **Bookhoard** on KOReader home screen
2. You'll see:
- **All Books**: Complete library view
- **Collections**: Books organized by collections
- **Recent**: Latest additions
3. Tap any category to browse
#### Download a Book
1. Browse to find a book
2. Tap the book to see details
3. Tap **Download**
4. Progress bar shows download status
5. Book opens automatically when complete
#### Download Entire Collections
1. In Bookhoard catalog, tap **Collections**
2. Select a collection
3. Tap **Download All** to get all books
4. Downloads queue and process in background
### OPDS Features
#### Supported Formats
KOReader OPDS supports:
### Supported Formats
- **EPUB**: Standard ebook format
- **KEPUB**: Kobo-optimized format (KOReader handles this well)
- **KEPUB**: Kobo-optimized format
- **PDF**: Fixed-layout documents
- **CBZ**: Comic book archives
- **TXT**: Plain text files
- **RTF**: Rich text format
#### Automatic Book Matching
Books downloaded via OPDS are automatically matched:
- Uses SHA-256 hashes for precise matching
- Falls back to title/author matching
- Links to your existing Bookhoard library
- Progress syncs automatically
#### Collection Integration
Your Bookhoard collections appear in KOReader:
- Collection **"To Read"** → KOReader category
- Collection **"Science Fiction"** → Browseable section
- Custom collections → Preserved organization
### KOReader OPDS Settings
#### Update Interval
Configure how often KOReader checks for new books:
1. KOReader menu → Tools → OPDS
2. Set **Update Interval**: 5min, 15min, 1hr, manual
3. **Recommended**: 15min for balance
#### Download Location
Choose where to store downloaded books:
1. KOReader menu → File Browser
2. Set **Default Download Folder**
3. **Recommended**: `/mnt/us/Documents/` (Kindle) or `/mnt/onboard/Documents/` (Kobo)
#### Auto-Download
Automatically download new books from collections:
1. KOReader menu → Tools → OPDS
2. Enable **Auto-Download New Books**
3. Select collections to monitor
4. New books download automatically when connected to Wi-Fi
### OPDS Troubleshooting
#### Catalog Not Loading
**Problem**: Bookhoard catalog shows error or won't load
**Solutions**:
1. Verify device is connected to Wi-Fi
2. Check OPDS URL is correct in settings
3. Try accessing OPDS URL in your browser
4. Ensure Bookhoard server is running
5. Check Bookhoard device is approved
#### Download Fails
**Problem**: Book download starts but fails
**Solutions**:
1. Check Wi-Fi signal strength
2. Ensure sufficient storage on device
3. Try downloading a smaller book
4. Check Bookhoard has the book file
5. Review Bookhoard logs for errors
#### Book Opens But Progress Doesn't Sync
**Problem**: Downloaded book doesn't sync progress
**Solutions**:
1. Verify book is matched to Bookhoard library
2. Check device sync settings are enabled
3. Try manual sync from device
4. Ensure book exists in Bookhoard with same hash
5. Check Bookhoard Progress page
#### Slow Downloads
**Problem**: Books take too long to download
**Solutions**:
1. Stay close to Wi-Fi router
2. Use 5GHz Wi-Fi if available
3. Close other apps using bandwidth
4. Download smaller books first
5. Consider USB for large books (100MB+)
### Advanced OPDS Configuration
#### Custom User-Agent
Some OPDS catalogs require specific user agent:
```lua
-- In KOReader settings
OPDSUserAgent = "KOReader/2024.01"
```
#### Authentication Token
If Bookhoard requires token authentication:
1. Get token from Bookhoard device settings
2. Add to OPDS URL: `?token=YOUR_TOKEN`
3. KOReader includes token in all requests
#### Compression
Enable compression for faster downloads:
```lua
-- In KOReader settings
OPDSCompressionEnabled = true
```
### OPDS vs USB Transfer
| Feature | OPDS (Wireless) | USB Transfer |
| ----------------- | ----------------------------- | ----------------------- |
| **Convenience** | ⭐⭐⭐⭐⭐ No cable needed | ⭐⭐ Requires cable |
| **Speed** | ⭐⭐⭐ Fast (Wi-Fi dependent) | ⭐⭐⭐⭐⭐ Very fast |
| **Bulk Transfer** | ⭐⭐⭐ One at a time | ⭐⭐⭐⭐⭐ Many at once |
| **Progress Sync** | ⭐⭐⭐⭐⭐ Instant | ⭐⭐⭐⭐ After transfer |
| **Accessibility** | ⭐⭐⭐⭐⭐ Anywhere | ⭐⭐ At computer only |
| **Reliability** | ⭐⭐⭐⭐ Very good | ⭐⭐⭐⭐⭐ Excellent |
**Recommendation**: Use OPDS for daily reading (convenience), USB for bulk library transfers.
### OPDS Tips and Tricks
1. **Favorite Collections**: Pin frequently-used collections to home screen
2. **Batch Downloads**: Start multiple downloads before leaving Wi-Fi
3. **Download Queue**: Downloads continue in background while reading
4. **Storage Management**: Check free space before downloading large collections
5. **Network Speed**: Use 5GHz Wi-Fi for faster downloads if available
Books downloaded via OPDS are automatically matched to your library, so their progress syncs from the first page.
## Troubleshooting
### Pending Registration Never Appears
**Problem**: You entered the Server URL, but no pending registration shows in Bookhoard
**Solutions**:
1. Verify the Server URL is correct (no trailing path — just the base address)
2. Make sure KOReader is connected to Wi-Fi
3. Check the Bookhoard server is reachable from the device's network
4. Registrations expire after 5 minutes — re-run the sync and approve quickly
### Connection Refused
**Problem**: "Connection refused" error
**Problem**: "Connection refused" error on the device
**Solutions**:
- Verify Bookhoard is running on your computer
- Check the server URL and port (8765)
- Ensure device is on same Wi-Fi network
- Try using your computer's IP address instead of "localhost"
- Verify Bookhoard is running
- Check the server address and port (default `8765`)
- Ensure the device is on the same Wi-Fi network as the server
- Use the server's LAN IP instead of `localhost`
### Authentication Failed
### Sync Not Working After Approval
**Problem**: "Authentication failed" error
**Problem**: Device shows as approved but changes don't appear in Bookhoard
**Solutions**:
- Verify username and password
- Check your account is active and not locked
- Try logging in to Bookhoard web interface first
- Reset password if needed
### Sync Not Working
**Problem**: Changes not appearing in Bookhoard
**Solutions**:
- Enable debug logging in KOReader
- Check Bookhoard Device Management page for errors
- Verify sync is enabled in KOReader settings
- Try manual sync to trigger immediate update
- Check Bookhoard logs for sync errors
- Trigger a manual sync from the plugin menu
- Check the device shows as enabled on the **Devices** page (open its settings from the icon next to the device)
- Verify the book appears as an unlinked book for the device and link it if needed
- Check Bookhoard server logs for errors
### Conflicts Detected
**Problem**: Sync conflicts when reading on multiple devices
**Problem**: Sync conflicts when reading the same book on multiple devices
**Solutions**:
1. Go to Bookhoard **Conflicts** page
2. Review conflicting progress from each device
1. Open the book's detail page and click **Sync Progress**, or open the Conflicts page (`/conflicts`)
2. Review the progress reported by each device
3. Choose which device's progress to keep
4. Set auto-resolution preference for future conflicts
### Large Files Not Syncing
**Problem**: Large annotations or highlights fail to sync
**Solutions**:
- Check Bookhoard sync queue for stuck items
- Increase sync timeout in KOReader settings
- Break up large highlights into smaller segments
- Verify network bandwidth is sufficient
## Security Best Practices
1. **Use HTTPS**: If deploying Bookhoard publicly, configure SSL/TLS
2. **Strong Password**: Use a secure password for your Bookhoard account
3. **Network Security**: Ensure your Wi-Fi network is secure (WPA2/WPA3)
4. **Device Authorization**: Only approve devices you recognize
5. **Regular Updates**: Keep KOReader updated to the latest version
1. **Use HTTPS**: If exposing Bookhoard beyond your LAN, configure SSL/TLS
2. **Network Security**: Ensure your Wi-Fi network is secure (WPA2/WPA3)
3. **Device Authorization**: Only approve pending registrations you initiated
4. **Revoke lost devices**: Remove devices you no longer use from the Devices page
## Additional Resources
- [KOReader Documentation](https://github.com/koreader/koreader)
- [KOReader Forum](https://www.mobileread.com/forums/forumdisplay.php?f=271)
- [Bookhoard Universal Sync Guide](../sync-guide.md)
- [Kobo Setup Guide](kobo-setup.md)
## Support
If you encounter issues:
1. Check the troubleshooting section above
2. Enable debug logging and review KOReader logs
3. Check Bookhoard sync queue and device management pages
4. Open an issue on the Bookhoard GitHub repository
- [Bookhoard KOReader Plugin](https://git.linuxhg.com/Bookhoard/bookhoard.koplugin)
---
**Last Updated**: 2026-01-31
**Bookhoard Version**: 1.0
**KOReader Version**: 2024.01+
**Last Updated**: August 2026
**Bookhoard Version**: 1.0
+28 -30
View File
@@ -1,60 +1,58 @@
## Saving Custom Filters
# Saving Custom Filters
The bookshelf page allows you to save custom filter presets for quick access.
The bookshelf (**All Books**) page lets you save custom filter presets for quick access.
### How to Save a Filter
## Bookshelf Toolbar
The All Books page has a toolbar with:
- A **search input** for quick text searches
- A **sort** dropdown (title, author, date added, page count)
- A **Filters** button that opens the filter drawer (author, tags, series, and more)
- **Save**, **Load**, and **Clear** buttons for filter presets
## How to Save a Filter
1. Navigate to the **All Books** page
2. Set your desired filters (genre, author, series, etc.)
3. Click the **💾 Save Filter** button
2. Click **Filters** to open the drawer, set your desired filters, and click **Apply Filters**
3. Click the **Save** button in the toolbar
4. Enter a name for your filter (e.g., "My Sci-Fi Books")
5. Click **Save**
### Loading Saved Filters
## Loading Saved Filters
After saving filters, you can quickly load them from the saved filters dropdown:
1. Click the **Load** button to open the **Saved Filters** dropdown
2. Click a filter's name to apply it
3. The filter values are applied instantly, without a page reload
1. Click the **📋 Saved Filters** button (next to the Save Filter button)
2. Select a filter from the dropdown list
3. The filter values are automatically applied to the form
4. Your books are instantly filtered to show matching results
## Managing Saved Filters
**Tips:**
- Saved filters appear in the dropdown with their names
- Hover over a filter to see a delete button (🗑️)
- Click a filter name to apply it instantly
- Filters are applied without page reload (instant feedback)
**Delete a filter:**
### Managing Saved Filters
**View Saved Filters:**
- Saved filters are displayed in the dropdown
- Each filter shows its name (e.g., "My Sci-Fi Books")
**Delete a Filter:**
1. Click the **📋 Saved Filters** button
2. Hover over the filter you want to delete
3. Click the **🗑️** delete button
4. Confirm deletion
5. The filter is removed from your list
1. Click the **Load** button to open the **Saved Filters** dropdown
2. Click the trash icon next to the filter you want to remove
3. Confirm deletion
**Filter Privacy:**
Saved filters are **private to your account**. Other users cannot see or modify your filters.
### Common Use Cases
## Common Use Cases
**Reading by Genre:**
1. Filter by genre: "Science Fiction"
1. Filter by tag: "Science Fiction"
2. Save as "Sci-Fi Books"
3. Quickly access all your sci-fi collection anytime
**Author Collections:**
1. Filter by author: "Isaac Asimov"
2. Save as "Asimov Books"
3. Switch between different author collections instantly
**Series Tracking:**
1. Filter by series: "Foundation"
2. Save as "Foundation Series"
3. Track your progress through a series
+9 -9
View File
@@ -6,10 +6,10 @@ Your profile contains your account information and preferences.
### How to Update
1. Click on your **username** (top-right)
2. Select **Profile** from the dropdown
1. Click on your **username** at the bottom of the sidebar to expand the account menu
2. Select **Profile**
3. Edit any fields in the "Account Information" section
4. Click **Update Profile**
4. Click **Save Changes**
5. Changes take effect immediately
### Fields You Can Update
@@ -65,7 +65,7 @@ When you delete your account:
1. Go to **Profile** page
2. Scroll to "Danger Zone" (bottom of page)
3. Click **Remove My Account**
4. Confirm by clicking "OK" in the popup
4. Confirm the deletion prompt
**Note:** If you're the last admin, you cannot delete your account for security reasons.
@@ -75,10 +75,12 @@ Personalize your reading experience with different color themes.
### Quick Theme Switch
1. Click the **paintbrush icon** (top-right, next to your username)
2. Select a theme from the dropdown
1. In the sidebar, open the **Appearance** panel (palette icon, near the bottom)
2. Select a theme from the list; your active theme is marked with a checkmark
3. Changes apply instantly
For the full theme list and bookshelf background (wood) options, see [Themes and Wood Paneling](themes.md).
### Available Themes
- **Tokyo Night** (default) - Blue/purple accents
@@ -88,9 +90,7 @@ Personalize your reading experience with different color themes.
- **Monokai** - Classic vibrant colors
- **One Dark Pro** - Atom editor inspired
- **Material Dark** - Google Material Design
- **Wood Light** - Light wood texture
- **Wood Dark** - Dark wood texture
- **Wood Mahogany** - Reddish-brown wood
- **Catppuccin Mocha / Macchiato / Frappé / Latte** - Soothing pastel palettes (Latte is light)
## For Admin Users
+5 -4
View File
@@ -13,10 +13,11 @@ Tags are keywords or categories assigned to books, such as:
### Filtering by Tags
1. Navigate to the **Bookshelf** page
2. Use the **Tags** filter input
3. Start typing to see autocomplete suggestions
4. Select a tag or press Enter to filter
1. Navigate to the **All Books** page
2. Click **Filters** in the toolbar to open the filter drawer
3. Use the **Tags** filter input
4. Start typing to see autocomplete suggestions
5. Select a tag or press Enter, then click **Apply Filters**
**Example:** Typing "Sci" will suggest "Science Fiction"
+43 -40
View File
@@ -21,7 +21,7 @@
🔄 **Automatic Sync** - Your reading progress syncs automatically when you turn pages
📱 **Multi-Platform** - Works with web browsers, KOReader, Kobo devices, and mobile apps
📱 **Multi-Platform** - Works with web browsers and KOReader, with native Kobo sync and mobile apps on the roadmap
📍 **Precise Location Tracking** - Supports EPUB CFI, page numbers, percentages, and character offsets
@@ -37,19 +37,24 @@
### Currently Supported ✅
| Platform | Status | Sync Method | Notes |
| ---------------- | ------------------ | --------------------------- | ------------------------------ |
| **Web Browser** | ✅ Fully Supported | Real-time WebSocket | Any modern browser |
| **KOReader** | ✅ Fully Supported | Wi-Fi (Calibre-compatible) | Kindle, Kobo, PocketBook, etc. |
| **Kobo Devices** | ✅ Fully Supported | Wi-Fi (Kobo API-compatible) | Clara, Libra, Sage, etc. |
| Platform | Status | Sync Method | Notes |
| ---------------- | ------------------ | --------------------------- | ---------------------------------- |
| **Web Browser** | ✅ Fully Supported | Real-time WebSocket | Any modern browser |
| **KOReader** | ✅ Fully Supported | Wi-Fi (Bookhoard plugin) | Kindle, Kobo, PocketBook hardware |
### Coming Soon 🚧
| Platform | Expected Release |
| --------------------- | ---------------- |
| **Mobile Apps** | Q2 2026 |
| **Kindle Devices** | Q3 2026 |
| **Remarkable Tablet** | Q4 2026 |
| Platform | Status |
| --------------------- | ------------------------------------------------------------- |
| **Kobo Devices** | Native sync coming soon — use KOReader on Kobo hardware today |
| **Mobile Apps** | Android/iOS apps coming later |
### On the Roadmap 🔭
| Platform | Status |
| --------------------- | ---------------------------------------- |
| **Kindle Devices** | Under consideration (no date yet) |
| **Remarkable Tablet** | Under consideration (no date yet) |
---
@@ -74,22 +79,21 @@
For detailed device configuration instructions, see the appropriate setup guide:
- **[Kobo Setup Guide](devices/kobo-setup.md)** - Kobo e-reader configuration
- **[KOReader Setup Guide](devices/koreader-setup.md)** - KOReader configuration
- **[KOReader Setup Guide](devices/koreader-setup.md)** - KOReader configuration (Kindle, Kobo, and PocketBook hardware)
- **[Kobo Setup Guide](devices/kobo-setup.md)** - Native Kobo sync (coming soon; use KOReader today)
### Quick Overview
**Registration Process**:
**Registration Process** (KOReader):
1. Register device in Bookhoard web interface (Settings → Devices)
2. Approve device via QR code or approval URL
3. Configure sync settings on your device
4. Start reading - progress syncs automatically!
1. Install the Bookhoard plugin and enter your server URL in KOReader
2. Approve the pending registration on the Bookhoard **Devices** page (sidebar navigation)
3. That's it — sync starts automatically once approved
**Device Management**:
```
Settings → Devices
Devices page (sidebar navigation)
```
You can:
@@ -124,7 +128,7 @@ Sometimes a book on your device can't be automatically matched to your library.
### Viewing Unlinked Books
```
Settings → Devices → Select Device → View Unlinked Books
Devices page select device → unlinked books
```
### Resolving Unlinked Books
@@ -284,7 +288,7 @@ This ensures your highlights work across all devices, even with different page c
**Solutions**:
1. Check device is online: `Settings → Devices`
1. Check device is online: Devices page (sidebar navigation)
2. Verify sync is enabled for the device
3. Check sync URL is correct
4. Ensure device has network connection
@@ -318,7 +322,7 @@ This ensures your highlights work across all devices, even with different page c
**Solutions**:
1. Go to `Settings → Conflicts`
1. Open the book's detail page and click **Sync Progress**, or go to the Conflicts page (`/conflicts`)
2. Review both device progress
3. Choose which device's progress to keep
4. Or choose "Merge" (keeps furthest progress)
@@ -331,7 +335,7 @@ This ensures your highlights work across all devices, even with different page c
1. Switch to checkpoint mode
2. Increase sync interval
3. Use Wi-Fi instead of cellular (for mobile)
3. Sync less frequently
---
@@ -341,7 +345,7 @@ This ensures your highlights work across all devices, even with different page c
**DO**:
- Use checkpoint mode when on cellular data
- Use checkpoint mode when on slow connections
- Keep device firmware updated
- Use Wi-Fi when available
- Approve only devices you own
@@ -369,7 +373,7 @@ This ensures your highlights work across all devices, even with different page c
- **Primary Device**: KOReader on e-reader
- **Secondary Device**: Web browser (work/home)
- **Mobile Device**: Phone app (commute)
- **On the go**: Web browser on a phone (dedicated mobile apps coming later)
**Sync Strategy**:
@@ -393,7 +397,7 @@ This ensures your highlights work across all devices, even with different page c
**Manual Resolution**:
```
Settings → Conflicts → Select conflictChoose winner
Book detail → Sync Progresschoose winner
```
**Options**:
@@ -408,7 +412,7 @@ Settings → Conflicts → Select conflict → Choose winner
**View Queue Status**:
```
Settings → Devices → Select Device → View Queue
Devices page → sync queue section
```
**Queue Stats**:
@@ -436,7 +440,7 @@ Settings → Devices → Select Device → View Queue
**View History**:
```
Book → Reading History
Progress page (sidebar navigation), or the book's detail page
```
**Privacy**:
@@ -503,9 +507,8 @@ Book → Reading History
### For Better Battery Life
1. **Checkpoint mode** - Fewer sync requests
2. **Wi-Fi only** - Disable cellular
3. **Increase sync interval** - Fewer updates
4. **Close when not reading** - Reduces background activity
2. **Increase sync interval** - Fewer updates
3. **Close when not reading** - Reduces background activity
---
@@ -523,7 +526,7 @@ A: No, devices are tied to individual accounts for security.
A: All sync data for that book is removed from the server.
**Q: Can I export my reading data?**
A: Yes! Settings → Export → Download sync data.
A: Reading data isn't exportable from the UI yet — it's accessible via the API.
**Q: Does sync work over the internet?**
A: Yes, if your server is publicly accessible with HTTPS.
@@ -537,7 +540,7 @@ A: Approximately 1KB per page turn, 50KB per annotation.
A: Uses percentage and EPUB CFI for universal positioning.
**Q: Can I sync with Calibre anymore?**
A: Yes! KOReader sync is Calibre-compatible.
A: Bookhoard's KOReader sync uses a dedicated plugin (server-side approval, device tokens) — no Calibre involvement required.
**Q: What if I lose my device?**
A: Revoke it in settings and register a new one.
@@ -571,18 +574,18 @@ A: Yes, HTTPS/TLS 1.3 for all sync traffic.
## Changelog
### Version 1.0.0 (January 2026)
### Version 1.0.x (2026)
- ✅ Initial release
- ✅ KOReader sync support
- ✅ Kobo device support
- ✅ Web sync support
- ✅ KOReader sync (progress, bookmarks, highlights, notes)
- ✅ Conflict resolution
- ✅ Offline queue
- ✅ Real-time WebSocket sync
- 🚧 Native Kobo sync (coming soon)
- 🚧 Mobile apps (coming later)
---
**Last Updated**: January 31, 2026
**Version**: 1.0.0
**License**: MIT
**Last Updated**: August 2026
**Version**: 1.0
**License**: AGPL-3.0
+8 -6
View File
@@ -18,10 +18,12 @@ Bookhoard includes multiple color themes to suit your preferences:
### Changing Your Theme
1. Click the theme icon (palette) in the header
2. Select your preferred color theme
1. In the sidebar, open the **Appearance** panel (palette icon, near the bottom)
2. Pick a theme from the list — each option shows its color swatch, and your active theme is marked with a checkmark
3. Your choice is saved automatically and synced across devices
On small screens, open the sidebar with the menu button in the top bar first.
## Wood Paneling
Wood paneling adds texture to your dashboard bookshelf background, giving it a classic bookshelf feel.
@@ -35,10 +37,10 @@ Wood paneling adds texture to your dashboard bookshelf background, giving it a c
### Applying Wood Paneling
1. Click the theme icon (palette) in the header
2. Scroll to "Bookshelf Background" section
3. Select your preferred wood texture
4. Texture is applied to dashboard bookshelf only
1. In the sidebar, open the **Appearance** panel (palette icon, near the bottom)
2. Scroll to the **Bookshelf** section below the theme list
3. Select your preferred wood texture (each option shows a texture swatch; **None** is the default)
4. Texture is applied to the dashboard bookshelf background
**Note:** Wood paneling is a browser preference and is not synced across devices.
+5 -7
View File
@@ -6,18 +6,16 @@ Welcome to the Bookhoard user documentation. This section contains guides for us
Learn how to configure your e-reader devices to sync with Bookhoard:
- **[Kobo Setup Guide](devices/kobo-setup.md)** - Complete guide for Kobo e-readers
- Device registration
- Sync configuration
- OPDS wireless book delivery
- Troubleshooting
- **[KOReader Setup Guide](devices/koreader-setup.md)** - Complete guide for KOReader
- Installation on Kindle/Kobo/PocketBook
- Sync setup
- Plugin setup with server-side device approval
- Progress, bookmark, highlight, and note sync
- OPDS catalog access
- Troubleshooting
- **[Kobo Setup Guide](devices/kobo-setup.md)** - Native Kobo sync (coming soon)
- In the meantime, KOReader works great on Kobo hardware
## 🔄 Sync Configuration
- **[Universal Sync Guide](sync-guide.md)** - Understanding and using sync features
+1 -1
View File
@@ -29,6 +29,7 @@ require (
github.com/yuin/goldmark v1.8.2
github.com/yuin/goldmark-highlighting v0.0.0-20220208100518-594be1970594
golang.org/x/crypto v0.50.0
golang.org/x/net v0.53.0
golang.org/x/text v0.36.0
)
@@ -57,7 +58,6 @@ require (
github.com/rogpeppe/go-internal v1.14.1 // indirect
github.com/xyproto/randomstring v1.2.0 // indirect
golang.org/x/image v0.39.0 // indirect
golang.org/x/net v0.53.0 // indirect
golang.org/x/sync v0.20.0 // indirect
golang.org/x/sys v0.43.0 // indirect
golang.org/x/time v0.15.0 // indirect
+8 -19
View File
@@ -45,30 +45,19 @@ func (c *Config) DatabaseURL() string {
c.DatabaseUser, c.DatabasePassword, c.DatabaseHost, c.DatabasePort, c.DatabaseName)
}
// GetBaseURL returns the base URL from system configuration database with fallback to config/env var
func GetBaseURL(ctx context.Context, db interface{}) string {
// Try to get from database first
type SystemConfigQuerier interface {
GetSystemConfig(ctx context.Context, key string) (SystemConfigRow, error)
}
// SystemConfigGetter returns the value for a system config key, or an error.
type SystemConfigGetter func(ctx context.Context, key string) (string, error)
if querier, ok := db.(SystemConfigQuerier); ok {
config, err := querier.GetSystemConfig(ctx, "base_url")
if err == nil && config.Value != "" {
return config.Value
}
// GetBaseURL returns the base URL from system configuration database, or empty
// string if not set. The getter abstraction avoids importing the database package.
func GetBaseURL(ctx context.Context, getter SystemConfigGetter) string {
val, err := getter(ctx, "base_url")
if err == nil && val != "" {
return val
}
// Fallback: return empty string - caller should use their own fallback
return ""
}
// SystemConfigRow represents a system configuration row
type SystemConfigRow struct {
Key string
Value string
}
func getEnv(key, defaultValue string) string {
if value := os.Getenv(key); value != "" {
return value
+21 -5
View File
@@ -92,6 +92,17 @@ type DictionaryCache struct {
AccessedAt pgtype.Timestamptz `db:"accessed_at" json:"accessed_at"`
}
type HashConflicts struct {
ID pgtype.UUID `db:"id" json:"id"`
LibraryID pgtype.UUID `db:"library_id" json:"library_id"`
FileSha256 string `db:"file_sha256" json:"file_sha256"`
Status string `db:"status" json:"status"`
Resolution pgtype.Text `db:"resolution" json:"resolution"`
ResolvedBy pgtype.UUID `db:"resolved_by" json:"resolved_by"`
CreatedAt pgtype.Timestamptz `db:"created_at" json:"created_at"`
ResolvedAt pgtype.Timestamptz `db:"resolved_at" json:"resolved_at"`
}
type KoboEntitlements struct {
ID pgtype.UUID `db:"id" json:"id"`
DeviceID pgtype.UUID `db:"device_id" json:"device_id"`
@@ -484,11 +495,16 @@ type SystemConfig struct {
}
type SystemSettings struct {
ID pgtype.UUID `db:"id" json:"id"`
SettingKey string `db:"setting_key" json:"setting_key"`
SettingValue string `db:"setting_value" json:"setting_value"`
Description pgtype.Text `db:"description" json:"description"`
UpdatedAt pgtype.Timestamptz `db:"updated_at" json:"updated_at"`
ID pgtype.UUID `db:"id" json:"id"`
SettingKey string `db:"setting_key" json:"setting_key"`
SettingValue string `db:"setting_value" json:"setting_value"`
Description pgtype.Text `db:"description" json:"description"`
UpdatedAt pgtype.Timestamptz `db:"updated_at" json:"updated_at"`
SettingType pgtype.Text `db:"setting_type" json:"setting_type"`
MinValue pgtype.Text `db:"min_value" json:"min_value"`
MaxValue pgtype.Text `db:"max_value" json:"max_value"`
RequiresRestart pgtype.Bool `db:"requires_restart" json:"requires_restart"`
Category pgtype.Text `db:"category" json:"category"`
}
type UnlinkedBooks struct {
+40 -1
View File
@@ -26,7 +26,7 @@ type Querier interface {
CheckForProgressConflicts(ctx context.Context, arg CheckForProgressConflictsParams) (int64, error)
// Cleanup expired OPDS tokens
CleanupExpiredOpdsTokens(ctx context.Context) error
CleanupExpiredRefreshTokens(ctx context.Context) error
CleanupExpiredRefreshTokens(ctx context.Context, dollar_1 float64) error
ClearDeviceSyncQueue(ctx context.Context, deviceID pgtype.UUID) error
ClearKoboShelf(ctx context.Context, deviceID pgtype.UUID) error
ClearKoboShelfByName(ctx context.Context, arg ClearKoboShelfByNameParams) error
@@ -53,6 +53,10 @@ type Querier interface {
// Create device shelf mapping
CreateDeviceShelfMapping(ctx context.Context, arg CreateDeviceShelfMappingParams) (DeviceShelfMappings, error)
CreateDictionaryEntry(ctx context.Context, arg CreateDictionaryEntryParams) (DictionaryCache, error)
// HASH CONFLICTS QUERIES
// Record a pending hash conflict (no-op if the group is already tracked, so
// resolved groups stay resolved and are never re-flagged)
CreateHashConflict(ctx context.Context, arg CreateHashConflictParams) error
// Libraries queries
CreateLibrary(ctx context.Context, arg CreateLibraryParams) (Libraries, error)
CreateMediaBookmark(ctx context.Context, arg CreateMediaBookmarkParams) (MediaBookmarks, error)
@@ -129,6 +133,8 @@ type Querier interface {
DeleteUnlinkedBook(ctx context.Context, id pgtype.UUID) error
DeleteUser(ctx context.Context, id pgtype.UUID) error
DeleteUserSystemCollection(ctx context.Context, arg DeleteUserSystemCollectionParams) error
// Find content-duplicate groups (same library + SHA-256, more than one row)
FindHashConflictGroups(ctx context.Context) ([]FindHashConflictGroupsRow, error)
GenerateKoboEntitlementId(ctx context.Context) (interface{}, error)
// ============================================
// ANNOTATION SERVE QUERIES
@@ -137,6 +143,7 @@ type Querier interface {
// Get all system config
GetAllSystemConfig(ctx context.Context) ([]SystemConfig, error)
GetAllSystemSettings(ctx context.Context) ([]GetAllSystemSettingsRow, error)
GetAllSystemSettingsFull(ctx context.Context) ([]SystemSettings, error)
GetAnnotationsForBook(ctx context.Context, arg GetAnnotationsForBookParams) ([]GetAnnotationsForBookRow, error)
GetBooksByTag(ctx context.Context, arg GetBooksByTagParams) ([]MediaItems, error)
// Get collection
@@ -183,6 +190,7 @@ type Querier interface {
GetFailedSyncQueueItems(ctx context.Context, limit int32) ([]SyncQueue, error)
GetFirstAdmin(ctx context.Context) (pgtype.UUID, error)
GetFirstAdminExclude(ctx context.Context, id pgtype.UUID) (GetFirstAdminExcludeRow, error)
GetHashConflict(ctx context.Context, id pgtype.UUID) (HashConflicts, error)
GetKoboEntitlementByContentId(ctx context.Context, arg GetKoboEntitlementByContentIdParams) (GetKoboEntitlementByContentIdRow, error)
GetKoboEntitlementByEntitlementId(ctx context.Context, arg GetKoboEntitlementByEntitlementIdParams) (GetKoboEntitlementByEntitlementIdRow, error)
GetKoboEntitlementsForDevice(ctx context.Context, deviceID pgtype.UUID) ([]GetKoboEntitlementsForDeviceRow, error)
@@ -231,12 +239,16 @@ type Querier interface {
GetMediaItemByOPFUUID(ctx context.Context, opfUuid pgtype.Text) (MediaItems, error)
// Get media item by SHA-256 hash
GetMediaItemBySHA256(ctx context.Context, fileSha256 pgtype.Text) (MediaItems, error)
// Get media item by SHA-256 hash within a specific library (content dedup)
GetMediaItemBySHA256AndLibrary(ctx context.Context, arg GetMediaItemBySHA256AndLibraryParams) (MediaItems, error)
// Get media item format by SHA-256
GetMediaItemFormatBySHA256(ctx context.Context, fileSha256 pgtype.Text) (MediaItemFormats, error)
// Get media item format by type
GetMediaItemFormatByType(ctx context.Context, arg GetMediaItemFormatByTypeParams) (MediaItemFormats, error)
// Get media item formats
GetMediaItemFormats(ctx context.Context, mediaItemID pgtype.UUID) ([]MediaItemFormats, error)
// Per-item user-data counts, used when choosing which duplicate copy to keep
GetMediaItemUsageCounts(ctx context.Context, mediaItemID pgtype.UUID) (GetMediaItemUsageCountsRow, error)
GetMediaNote(ctx context.Context, id pgtype.UUID) (MediaNotes, error)
// ============================================
// ANNOTATION SYNC QUERIES (notes)
@@ -278,6 +290,7 @@ type Querier interface {
GetSystemConfig(ctx context.Context, key string) (SystemConfig, error)
// System Settings queries
GetSystemSetting(ctx context.Context, settingKey string) (string, error)
GetSystemSettingFull(ctx context.Context, settingKey string) (SystemSettings, error)
GetSystemTimezone(ctx context.Context) (string, error)
GetTombstonedAnnotationsForBook(ctx context.Context, arg GetTombstonedAnnotationsForBookParams) ([]GetTombstonedAnnotationsForBookRow, error)
// Get universal progress for a book
@@ -303,6 +316,7 @@ type Querier interface {
// Get user reading history for analytics
GetUserReadingHistory(ctx context.Context, arg GetUserReadingHistoryParams) ([]GetUserReadingHistoryRow, error)
GetUserVisibleLibraries(ctx context.Context, userID pgtype.UUID) ([]GetUserVisibleLibrariesRow, error)
GetVisibleLibraryMediaCounts(ctx context.Context, userID pgtype.UUID) ([]GetVisibleLibraryMediaCountsRow, error)
HasRecentConflictResolution(ctx context.Context, arg HasRecentConflictResolutionParams) (bool, error)
IncrementSyncQueueAttempts(ctx context.Context, arg IncrementSyncQueueAttemptsParams) (SyncQueue, error)
// Check if book is in collection
@@ -313,12 +327,25 @@ type Querier interface {
ListAllConflictsByUserAndStatus(ctx context.Context, arg ListAllConflictsByUserAndStatusParams) ([]ListAllConflictsByUserAndStatusRow, error)
ListAllSyncQueueItems(ctx context.Context, arg ListAllSyncQueueItemsParams) ([]ListAllSyncQueueItemsRow, error)
ListConflictsByUser(ctx context.Context, userID pgtype.UUID) ([]ListConflictsByUserRow, error)
// ============================================
// ANNOTATION HISTORY (deleted-annotation archive)
// ============================================
// Lists every currently-tombstoned annotation for a book regardless of the
// tombstone TTL: this backs the book page's "recently deleted" history where
// users can restore or permanently remove entries. Rows whose tombstones have
// been purged by the daily maintenance sweep no longer exist at all.
ListDeletedAnnotationsForBook(ctx context.Context, arg ListDeletedAnnotationsForBookParams) ([]ListDeletedAnnotationsForBookRow, error)
ListDevicesByType(ctx context.Context, deviceType string) ([]Devices, error)
ListDevicesByUser(ctx context.Context, userID pgtype.UUID) ([]Devices, error)
ListLibraries(ctx context.Context) ([]ListLibrariesRow, error)
ListMediaItems(ctx context.Context, arg ListMediaItemsParams) ([]ListMediaItemsRow, error)
ListMediaItemsByLibrary(ctx context.Context, libraryID pgtype.UUID) ([]ListMediaItemsByLibraryRow, error)
// List all media items sharing a SHA-256 hash within a library (hash conflict group)
ListMediaItemsBySHA256AndLibrary(ctx context.Context, arg ListMediaItemsBySHA256AndLibraryParams) ([]MediaItems, error)
// List media items that have no stored SHA-256 (imported before hashing existed)
ListMediaItemsMissingHash(ctx context.Context) ([]MediaItems, error)
ListMediaItemsSorted(ctx context.Context, arg ListMediaItemsSortedParams) ([]ListMediaItemsSortedRow, error)
ListPendingHashConflicts(ctx context.Context) ([]ListPendingHashConflictsRow, error)
ListPendingSyncQueueItems(ctx context.Context, arg ListPendingSyncQueueItemsParams) ([]SyncQueue, error)
ListProcessingIssuesByLibrary(ctx context.Context, libraryID pgtype.UUID) ([]ListProcessingIssuesByLibraryRow, error)
ListSyncConflictsByMediaItem(ctx context.Context, arg ListSyncConflictsByMediaItemParams) ([]SyncConflicts, error)
@@ -329,6 +356,11 @@ type Querier interface {
PurgeExpiredBookmarkTombstones(ctx context.Context, deletedAt pgtype.Timestamptz) error
PurgeExpiredHighlightTombstones(ctx context.Context, deletedAt pgtype.Timestamptz) error
PurgeExpiredNoteTombstones(ctx context.Context, deletedAt pgtype.Timestamptz) error
PurgeMediaBookmarkByID(ctx context.Context, arg PurgeMediaBookmarkByIDParams) (int64, error)
// Permanent removal from the history (distinct from the TTL-driven purge,
// which is maintenance). Scoped to the owning user and book.
PurgeMediaHighlightByID(ctx context.Context, arg PurgeMediaHighlightByIDParams) (int64, error)
PurgeMediaNoteByID(ctx context.Context, arg PurgeMediaNoteByIDParams) (int64, error)
// Query media items by multiple identifiers with confidence scoring
QueryMediaItemsByIdentifiers(ctx context.Context, arg QueryMediaItemsByIdentifiersParams) ([]QueryMediaItemsByIdentifiersRow, error)
ReassignLibraries(ctx context.Context, arg ReassignLibrariesParams) error
@@ -336,11 +368,17 @@ type Querier interface {
// Remove book from collection
RemoveBookFromCollection(ctx context.Context, arg RemoveBookFromCollectionParams) error
RemoveBookFromKoboShelf(ctx context.Context, arg RemoveBookFromKoboShelfParams) error
// Re-parent all child rows of p_source onto p_target (defined in schema.sql)
ReparentMediaItemChildren(ctx context.Context, arg ReparentMediaItemChildrenParams) error
ResetSystemCollectionMetadata(ctx context.Context, arg ResetSystemCollectionMetadataParams) error
ResolveHashConflict(ctx context.Context, arg ResolveHashConflictParams) error
ResolveProcessingIssue(ctx context.Context, arg ResolveProcessingIssueParams) (ProcessingIssues, error)
ResolveSyncConflict(ctx context.Context, arg ResolveSyncConflictParams) (SyncConflicts, error)
// Resolve unlinked book
ResolveUnlinkedBook(ctx context.Context, arg ResolveUnlinkedBookParams) (UnlinkedBooks, error)
RestoreMediaBookmarkByID(ctx context.Context, arg RestoreMediaBookmarkByIDParams) (int64, error)
RestoreMediaHighlightByID(ctx context.Context, arg RestoreMediaHighlightByIDParams) (int64, error)
RestoreMediaNoteByID(ctx context.Context, arg RestoreMediaNoteByIDParams) (int64, error)
RevokeAllUserRefreshTokens(ctx context.Context, userID pgtype.UUID) error
RevokeDevice(ctx context.Context, id pgtype.UUID) error
// Revoke OPDS token
@@ -427,6 +465,7 @@ type Querier interface {
UpsertDashboardPreferences(ctx context.Context, arg UpsertDashboardPreferencesParams) (UserDashboardPreferences, error)
UpsertPanelData(ctx context.Context, arg UpsertPanelDataParams) (PanelData, error)
UpsertReaderSettings(ctx context.Context, arg UpsertReaderSettingsParams) (ReaderSettings, error)
UpsertSystemSetting(ctx context.Context, arg UpsertSystemSettingParams) (SystemSettings, error)
}
var _ Querier = (*Queries)(nil)
File diff suppressed because it is too large Load Diff
+205 -8
View File
@@ -137,10 +137,19 @@ LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = $1
WHERE COALESCE(lv.is_visible, true) = true
ORDER BY l.created_at ASC;
-- name: GetVisibleLibraryMediaCounts :many
SELECT l.id, COUNT(mi.id) as media_count
FROM libraries l
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = $1
LEFT JOIN media_items mi ON mi.library_id = l.id
WHERE COALESCE(lv.is_visible, true) = true
GROUP BY l.id;
-- Media Items queries
-- name: CreateMediaItem :one
INSERT INTO media_items (library_id, title, author, isbn, description, file_path, file_size, mime_type, cover_image_path, series, series_number, tags, tags_search, asin, date_published, publisher, contributors, contributors_search, language, edition, page_count, genre, copyright_year, goodreads_id, openlibrary_id, google_books_id, added_by_admin_id, created_at, imported_at, manga_type, reading_direction, series_count, volume, imprint, age_rating, web_url, story_arc, is_black_and_white, metadata_notes, community_rating, alternate_info, scan_information, summary, library_type_name)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35, $36, $37, $38, $39, $40, $41, $42, $43, $44)
ON CONFLICT (library_id, file_path) DO UPDATE SET updated_at = NOW()
RETURNING *;
-- name: GetMediaItem :one
@@ -365,9 +374,29 @@ SELECT setting_value FROM system_settings WHERE setting_key = $1;
-- name: UpdateSystemSetting :exec
UPDATE system_settings SET setting_value = $2, updated_at = NOW() WHERE setting_key = $1;
-- name: UpsertSystemSetting :one
INSERT INTO system_settings (setting_key, setting_value, description, setting_type, min_value, max_value, requires_restart, category)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
ON CONFLICT (setting_key) DO UPDATE
SET setting_value = EXCLUDED.setting_value,
description = EXCLUDED.description,
setting_type = EXCLUDED.setting_type,
min_value = EXCLUDED.min_value,
max_value = EXCLUDED.max_value,
requires_restart = EXCLUDED.requires_restart,
category = EXCLUDED.category,
updated_at = NOW()
RETURNING *;
-- name: GetSystemSettingFull :one
SELECT * FROM system_settings WHERE setting_key = $1;
-- name: GetAllSystemSettings :many
SELECT setting_key, setting_value, description FROM system_settings ORDER BY setting_key;
-- name: GetAllSystemSettingsFull :many
SELECT * FROM system_settings ORDER BY category, setting_key;
-- name: CreateMediaRating :one
INSERT INTO media_ratings (media_item_id, user_id, rating)
VALUES ($1, $2, $3)
@@ -773,7 +802,9 @@ UPDATE media_highlights SET
last_modified_at = $12,
last_modified_source = $13,
device_sync_data = $14,
updated_at = NOW()
updated_at = NOW(),
deleted = FALSE,
deleted_at = NULL
WHERE id = $1
RETURNING *;
@@ -828,7 +859,9 @@ UPDATE media_notes SET
last_modified_at = $10,
last_modified_source = $11,
device_sync_data = $12,
updated_at = NOW()
updated_at = NOW(),
deleted = FALSE,
deleted_at = NULL
WHERE id = $1
RETURNING *;
@@ -884,7 +917,9 @@ UPDATE media_bookmarks SET
last_modified_at = $11,
last_modified_source = $12,
device_sync_data = $13,
created_at = created_at
created_at = created_at,
deleted = FALSE,
deleted_at = NULL
WHERE id = $1
RETURNING *;
@@ -957,7 +992,11 @@ SELECT
mh.dedup_key,
'highlight' as annotation_type,
mh.device_sync_data,
mh.deleted_at
mh.deleted_at,
mh.start_position,
mh.end_position,
mh.epubcfi_start,
mh.epubcfi_end
FROM media_highlights mh
WHERE mh.media_item_id = $1 AND mh.user_id = $2 AND mh.deleted = TRUE AND mh.deleted_at > $3
UNION ALL
@@ -966,7 +1005,11 @@ SELECT
mn.dedup_key,
'note' as annotation_type,
mn.device_sync_data,
mn.deleted_at
mn.deleted_at,
mn.position as start_position,
NULL as end_position,
mn.epubcfi_location as epubcfi_start,
NULL as epubcfi_end
FROM media_notes mn
WHERE mn.media_item_id = $1 AND mn.user_id = $2 AND mn.deleted = TRUE AND mn.deleted_at > $3
UNION ALL
@@ -975,11 +1018,96 @@ SELECT
mb.dedup_key,
'bookmark' as annotation_type,
mb.device_sync_data,
mb.deleted_at
mb.deleted_at,
mb.position as start_position,
NULL as end_position,
mb.cfi_position as epubcfi_start,
NULL as epubcfi_end
FROM media_bookmarks mb
WHERE mb.media_item_id = $1 AND mb.user_id = $2 AND mb.deleted = TRUE AND mb.deleted_at > $3
ORDER BY deleted_at DESC;
-- ============================================
-- ANNOTATION HISTORY (deleted-annotation archive)
-- ============================================
-- Lists every currently-tombstoned annotation for a book regardless of the
-- tombstone TTL: this backs the book page's "recently deleted" history where
-- users can restore or permanently remove entries. Rows whose tombstones have
-- been purged by the daily maintenance sweep no longer exist at all.
-- name: ListDeletedAnnotationsForBook :many
SELECT
mh.id,
mh.dedup_key,
'highlight' as annotation_type,
mh.selection_text as display_text,
mh.note_text as secondary_text,
mh.color,
mh.deleted_at,
mh.created_at
FROM media_highlights mh
WHERE mh.media_item_id = $1 AND mh.user_id = $2 AND mh.deleted = TRUE
UNION ALL
SELECT
mn.id,
mn.dedup_key,
'note' as annotation_type,
mn.content as display_text,
NULL::text as secondary_text,
NULL::text as color,
mn.deleted_at,
mn.created_at
FROM media_notes mn
WHERE mn.media_item_id = $1 AND mn.user_id = $2 AND mn.deleted = TRUE
UNION ALL
SELECT
mb.id,
mb.dedup_key,
'bookmark' as annotation_type,
mb.title as display_text,
mb.notes as secondary_text,
NULL::text as color,
mb.deleted_at,
mb.created_at
FROM media_bookmarks mb
WHERE mb.media_item_id = $1 AND mb.user_id = $2 AND mb.deleted = TRUE
ORDER BY deleted_at DESC;
-- name: RestoreMediaHighlightByID :execrows
UPDATE media_highlights SET
deleted = FALSE,
deleted_at = NULL,
last_modified_at = NOW()
WHERE id = $1 AND user_id = $2 AND media_item_id = $3 AND deleted = TRUE;
-- name: RestoreMediaNoteByID :execrows
UPDATE media_notes SET
deleted = FALSE,
deleted_at = NULL,
last_modified_at = NOW()
WHERE id = $1 AND user_id = $2 AND media_item_id = $3 AND deleted = TRUE;
-- name: RestoreMediaBookmarkByID :execrows
UPDATE media_bookmarks SET
deleted = FALSE,
deleted_at = NULL,
last_modified_at = NOW()
WHERE id = $1 AND user_id = $2 AND media_item_id = $3 AND deleted = TRUE;
-- Permanent removal from the history (distinct from the TTL-driven purge,
-- which is maintenance). Scoped to the owning user and book.
-- name: PurgeMediaHighlightByID :execrows
DELETE FROM media_highlights
WHERE id = $1 AND user_id = $2 AND media_item_id = $3 AND deleted = TRUE;
-- name: PurgeMediaNoteByID :execrows
DELETE FROM media_notes
WHERE id = $1 AND user_id = $2 AND media_item_id = $3 AND deleted = TRUE;
-- name: PurgeMediaBookmarkByID :execrows
DELETE FROM media_bookmarks
WHERE id = $1 AND user_id = $2 AND media_item_id = $3 AND deleted = TRUE;
-- Refresh Tokens queries
-- name: CreateRefreshToken :one
INSERT INTO refresh_tokens (user_id, token, expires_at)
@@ -999,7 +1127,7 @@ UPDATE refresh_tokens SET revoked_at = NOW() WHERE token = $1;
UPDATE refresh_tokens SET revoked_at = NOW() WHERE user_id = $1 AND revoked_at IS NULL;
-- name: CleanupExpiredRefreshTokens :exec
DELETE FROM refresh_tokens WHERE expires_at < NOW() OR (revoked_at IS NOT NULL AND revoked_at < NOW() - INTERVAL '7 days');
DELETE FROM refresh_tokens WHERE expires_at < NOW() OR (revoked_at IS NOT NULL AND revoked_at < NOW() - make_interval(secs => $1::double precision));
-- ============================================
-- FORMAT DETECTION & PROGRESS
@@ -1679,6 +1807,70 @@ RETURNING *;
-- name: GetMediaItemBySHA256 :one
SELECT * FROM media_items WHERE file_sha256 = $1;
-- Get media item by SHA-256 hash within a specific library (content dedup)
-- name: GetMediaItemBySHA256AndLibrary :one
SELECT * FROM media_items WHERE file_sha256 = $1 AND library_id = $2;
-- List all media items sharing a SHA-256 hash within a library (hash conflict group)
-- name: ListMediaItemsBySHA256AndLibrary :many
SELECT * FROM media_items WHERE file_sha256 = $1 AND library_id = $2 ORDER BY file_path;
-- List media items that have no stored SHA-256 (imported before hashing existed)
-- name: ListMediaItemsMissingHash :many
SELECT * FROM media_items WHERE file_sha256 IS NULL ORDER BY created_at;
-- Find content-duplicate groups (same library + SHA-256, more than one row)
-- name: FindHashConflictGroups :many
SELECT library_id, file_sha256, COUNT(*) AS dup_count
FROM media_items
WHERE file_sha256 IS NOT NULL
GROUP BY library_id, file_sha256
HAVING COUNT(*) > 1;
-- Per-item user-data counts, used when choosing which duplicate copy to keep
-- name: GetMediaItemUsageCounts :one
SELECT
(SELECT COUNT(*) FROM reading_progress rp WHERE rp.media_item_id = $1) AS progress_count,
(SELECT COUNT(*) FROM media_highlights mh WHERE mh.media_item_id = $1) AS highlights_count,
(SELECT COUNT(*) FROM media_bookmarks mb WHERE mb.media_item_id = $1) AS bookmarks_count,
(SELECT COUNT(*) FROM media_notes mn WHERE mn.media_item_id = $1) AS notes_count,
(SELECT COUNT(*) FROM collection_items ci WHERE ci.media_item_id = $1) AS collections_count;
-- HASH CONFLICTS QUERIES
-- Record a pending hash conflict (no-op if the group is already tracked, so
-- resolved groups stay resolved and are never re-flagged)
-- name: CreateHashConflict :exec
INSERT INTO hash_conflicts (library_id, file_sha256)
VALUES ($1, $2)
ON CONFLICT (library_id, file_sha256) DO NOTHING;
-- name: ListPendingHashConflicts :many
SELECT hc.id, hc.library_id, hc.file_sha256, hc.created_at,
l.name AS library_name,
COUNT(mi.id) AS item_count
FROM hash_conflicts hc
JOIN libraries l ON l.id = hc.library_id
LEFT JOIN media_items mi ON mi.library_id = hc.library_id AND mi.file_sha256 = hc.file_sha256
WHERE hc.status = 'pending'
GROUP BY hc.id, hc.library_id, hc.file_sha256, hc.created_at, l.name
ORDER BY hc.created_at;
-- name: GetHashConflict :one
SELECT * FROM hash_conflicts WHERE id = $1;
-- name: ResolveHashConflict :exec
UPDATE hash_conflicts
SET status = 'resolved',
resolution = $2,
resolved_by = $3,
resolved_at = NOW()
WHERE id = $1;
-- Re-parent all child rows of p_source onto p_target (defined in schema.sql)
-- name: ReparentMediaItemChildren :exec
SELECT reparent_media_item_children($1::uuid, $2::uuid);
-- Get media item by OPF identifier
-- name: GetMediaItemByOPFIdentifier :one
SELECT * FROM media_items WHERE opf_identifier = $1;
@@ -1726,6 +1918,11 @@ ORDER BY confidence_score DESC;
-- name: CreateMediaItemFormat :one
INSERT INTO media_item_formats (media_item_id, format_type, file_path, file_sha256, file_size_bytes, mime_type, converted_from_format_id)
VALUES ($1, $2, $3, $4, $5, $6, $7)
ON CONFLICT (media_item_id, format_type) DO UPDATE SET
file_path = EXCLUDED.file_path,
file_sha256 = EXCLUDED.file_sha256,
file_size_bytes = EXCLUDED.file_size_bytes,
mime_type = EXCLUDED.mime_type
RETURNING *;
-- Get media item formats
@@ -2383,7 +2580,7 @@ SET
title = $2,
notes = $3,
position = $4,
updated_at = NOW()
last_modified_at = NOW()
WHERE id = $1 AND user_id = $5
RETURNING *;
+342
View File
@@ -0,0 +1,342 @@
package database
// SettingsRegistry provides a typed, cached view over the system_settings table.
// It is the single source of truth for tunable runtime values that used to be
// hardcoded as Go literals.
//
// Consumers call the domain-specific getters (SessionDuration, OpdsPageSize,
// etc.) which read from an in-memory cache. The cache is populated by Load at
// startup and refreshed by Reload whenever a setting is written. Getters always
// fall back to a compiled-in default if the DB value is missing or unparsable,
// so a corrupt or deleted row can never break the app.
//
// SettingsRegistry lives in the database package (rather than its own package)
// so that every consumer already imports database and does not need to take on
// a new package import.
import (
"context"
"log"
"strconv"
"sync"
"time"
)
// SettingType enumerates the value types stored in system_settings.setting_type.
const (
SettingTypeInt = "int"
SettingTypeBool = "bool"
SettingTypeString = "string"
SettingTypeStringList = "string_list"
)
// SecondsPerDay / SecondsPerHour are conversion helpers used by defaults.
const (
SecondsPerMinute = 60
SecondsPerHour = 3600
SecondsPerDay = 86400
)
// SettingDefault holds the fallback value for a key. These mirror the literals that
// were previously hardcoded in the source so an empty/corrupt DB row preserves
// prior behavior exactly.
type SettingDefault struct {
Key string
Value string
Type string
Min string
Max string
RequiresRestart bool
Category string
Group string
Description string
}
// SettingDefaults is the source of truth for fallback values and metadata. New keys
// must be added here AND seeded in database/schema/schema.sql. Entries are ordered
// by (RequiresRestart, Group) so the admin UI renders coherent sub-sections.
var SettingDefaults = []SettingDefault{
{Key: "scan_poll_interval_seconds", Value: "60", Type: SettingTypeInt, Min: "1", Max: "3600", Category: "scanner", Group: "Scanning", Description: "How often to scan all libraries (seconds)"},
{Key: "auto_scan_enabled", Value: "true", Type: SettingTypeBool, Category: "scanner", Group: "Scanning", Description: "Whether auto-scanning is enabled system-wide"},
{Key: "default_timezone", Value: "UTC", Type: SettingTypeString, Category: "general", Group: "System Defaults", Description: "System default timezone"},
{Key: "session_duration_seconds", Value: "604800", Type: SettingTypeInt, Min: "300", Max: "31536000", Category: "security", Group: "Session", Description: "How long a login session stays valid"},
{Key: "password_min_length", Value: "8", Type: SettingTypeInt, Min: "1", Max: "128", Category: "security", Group: "Password Quality", Description: "Minimum password length"},
{Key: "password_require_upper", Value: "true", Type: SettingTypeBool, Category: "security", Group: "Password Quality", Description: "Require at least one uppercase letter (A-Z)"},
{Key: "password_require_lower", Value: "true", Type: SettingTypeBool, Category: "security", Group: "Password Quality", Description: "Require at least one lowercase letter (a-z)"},
{Key: "password_require_number", Value: "true", Type: SettingTypeBool, Category: "security", Group: "Password Quality", Description: "Require at least one number (0-9)"},
{Key: "password_require_special", Value: "true", Type: SettingTypeBool, Category: "security", Group: "Password Quality", Description: "Require at least one special character"},
{Key: "opds_default_page_size", Value: "50", Type: SettingTypeInt, Min: "1", Max: "500", Category: "api", Group: "OPDS Catalog", Description: "Default OPDS page size"},
{Key: "opds_max_page_size", Value: "200", Type: SettingTypeInt, Min: "1", Max: "1000", Category: "api", Group: "OPDS Catalog", Description: "Maximum OPDS page size"},
{Key: "device_rate_sync_per_min", Value: "60", Type: SettingTypeInt, Min: "1", Max: "10000", Category: "api", Group: "Device Rate Limits", Description: "Device sync requests per minute"},
{Key: "device_rate_progress_per_min", Value: "120", Type: SettingTypeInt, Min: "1", Max: "10000", Category: "api", Group: "Device Rate Limits", Description: "Device progress requests per minute"},
{Key: "device_rate_metadata_per_min", Value: "30", Type: SettingTypeInt, Min: "1", Max: "10000", Category: "api", Group: "Device Rate Limits", Description: "Device metadata requests per minute"},
{Key: "annotation_tombstone_ttl_days", Value: "30", Type: SettingTypeInt, Min: "1", Max: "3650", Category: "sync", Group: "Annotation Retention", Description: "How long deleted annotations are kept before purge"},
{Key: "conversion_cache_ttl_hours", Value: "24", Type: SettingTypeInt, Min: "1", Max: "720", Category: "performance", Group: "Conversion Cache", Description: "How long converted (kepub) files are cached"},
{Key: "auth_rate_limit_per_min", Value: "10", Type: SettingTypeInt, Min: "1", Max: "10000", RequiresRestart: true, Category: "security", Group: "Auth Rate Limiting", Description: "Global auth API rate limit (requests per minute)"},
{Key: "login_max_attempts", Value: "5", Type: SettingTypeInt, Min: "1", Max: "100", RequiresRestart: true, Category: "security", Group: "Login Lockout", Description: "Failed login attempts before lockout"},
{Key: "login_lockout_minutes", Value: "15", Type: SettingTypeInt, Min: "1", Max: "10080", RequiresRestart: true, Category: "security", Group: "Login Lockout", Description: "Lockout duration after too many failed logins"},
{Key: "sync_queue_interval_seconds", Value: "5", Type: SettingTypeInt, Min: "1", Max: "3600", RequiresRestart: true, Category: "sync", Group: "Sync Queue", Description: "How often the sync queue flushes"},
{Key: "sync_queue_batch_size", Value: "50", Type: SettingTypeInt, Min: "1", Max: "10000", RequiresRestart: true, Category: "sync", Group: "Sync Queue", Description: "Maximum items processed per sync queue flush"},
{Key: "worker_pool_size", Value: "3", Type: SettingTypeInt, Min: "1", Max: "100", RequiresRestart: true, Category: "performance", Group: "Worker Pool", Description: "Number of background worker goroutines"},
{Key: "worker_queue_cap", Value: "100", Type: SettingTypeInt, Min: "1", Max: "10000", RequiresRestart: true, Category: "performance", Group: "Worker Pool", Description: "Background worker job queue capacity"},
}
// defaultBy indexes SettingDefaults by key for O(1) lookup.
var defaultBy = func() map[string]SettingDefault {
m := make(map[string]SettingDefault, len(SettingDefaults))
for _, d := range SettingDefaults {
m[d.Key] = d
}
return m
}()
// Registry caches system_settings values in memory. The zero value is not
// usable; construct with New.
type SettingsRegistry struct {
q *Queries
mu sync.RWMutex
values map[string]string
loadedAt time.Time
}
// New returns a Registry backed by the given queries. The cache is empty
// until Load is called.
func NewSettingsRegistry(q *Queries) *SettingsRegistry {
return &SettingsRegistry{q: q, values: make(map[string]string)}
}
// Load populates the cache from the database. Missing rows fall back to the
// compiled defaults. Safe to call multiple times.
func (r *SettingsRegistry) Load(ctx context.Context) error {
rows, err := r.q.GetAllSystemSettings(ctx)
if err != nil {
return err
}
fresh := make(map[string]string, len(SettingDefaults))
for _, d := range SettingDefaults {
fresh[d.Key] = d.Value
}
for _, row := range rows {
if _, ok := fresh[row.SettingKey]; ok {
fresh[row.SettingKey] = row.SettingValue
}
}
r.mu.Lock()
r.values = fresh
r.loadedAt = time.Now()
r.mu.Unlock()
return nil
}
// Reload refreshes the cache from the database. Should be called after any
// setting write. On error the cache is left untouched and the error is logged.
func (r *SettingsRegistry) Reload(ctx context.Context) {
if err := r.Load(ctx); err != nil {
log.Printf("settings: reload failed: %v", err)
}
}
// raw returns the cached string value for a key (or the default), clamped to
// [min, max] for int-typed keys.
func (r *SettingsRegistry) raw(key string) string {
r.mu.RLock()
v, ok := r.values[key]
r.mu.RUnlock()
if !ok || v == "" {
v = defaultBy[key].Value
}
return v
}
func (r *SettingsRegistry) getInt(key string) int {
d := defaultBy[key]
v := r.raw(key)
n, err := strconv.Atoi(v)
if err != nil {
n, _ = strconv.Atoi(d.Value)
}
if d.Min != "" {
if mn, err := strconv.Atoi(d.Min); err == nil && n < mn {
n = mn
}
}
if d.Max != "" {
if mx, err := strconv.Atoi(d.Max); err == nil && n > mx {
n = mx
}
}
return n
}
func (r *SettingsRegistry) getBool(key string) bool {
v := r.raw(key)
b, err := strconv.ParseBool(v)
if err != nil {
b, _ = strconv.ParseBool(defaultBy[key].Value)
}
return b
}
// ---- Domain-specific getters (call sites use these) ----
// ScanPollInterval is how often the scanner polls, as a duration.
func (r *SettingsRegistry) ScanPollInterval() time.Duration {
return time.Duration(r.getInt("scan_poll_interval_seconds")) * time.Second
}
// AutoScanEnabled reports whether auto-scanning is on.
func (r *SettingsRegistry) AutoScanEnabled() bool { return r.getBool("auto_scan_enabled") }
// DefaultTimezone returns the configured default timezone name.
func (r *SettingsRegistry) DefaultTimezone() string { return r.raw("default_timezone") }
// SessionDuration is how long a login session / refresh token stays valid.
func (r *SettingsRegistry) SessionDuration() time.Duration {
return time.Duration(r.getInt("session_duration_seconds")) * time.Second
}
// PasswordMinLength is the minimum password length.
func (r *SettingsRegistry) PasswordMinLength() int { return r.getInt("password_min_length") }
// PasswordRules bundles the active complexity requirements.
type PasswordRules struct {
MinLength int
Upper bool
Lower bool
Number bool
Special bool
}
// PasswordRules returns the active password complexity configuration.
func (r *SettingsRegistry) PasswordRules() PasswordRules {
return PasswordRules{
MinLength: r.PasswordMinLength(),
Upper: r.getBool("password_require_upper"),
Lower: r.getBool("password_require_lower"),
Number: r.getBool("password_require_number"),
Special: r.getBool("password_require_special"),
}
}
// AuthRateLimit is the global auth endpoint rate limit (requests/minute). Read
// once at startup.
func (r *SettingsRegistry) AuthRateLimit() int { return r.getInt("auth_rate_limit_per_min") }
// LoginLockout returns (max attempts, lockout duration). Read once at startup.
func (r *SettingsRegistry) LoginLockout() (int, time.Duration) {
return r.getInt("login_max_attempts"), time.Duration(r.getInt("login_lockout_minutes")) * time.Minute
}
// OpdsDefaultPageSize is the default OPDS items-per-page.
func (r *SettingsRegistry) OpdsDefaultPageSize() int { return r.getInt("opds_default_page_size") }
// OpdsMaxPageSize is the maximum items-per-page a client may request.
func (r *SettingsRegistry) OpdsMaxPageSize() int { return r.getInt("opds_max_page_size") }
// DeviceRateLimits bundles the per-route device rate limits (requests/minute).
type DeviceRateLimits struct {
Sync int
Progress int
Metadata int
}
// DeviceRateLimits returns the active device rate limits.
func (r *SettingsRegistry) DeviceRateLimits() DeviceRateLimits {
return DeviceRateLimits{
Sync: r.getInt("device_rate_sync_per_min"),
Progress: r.getInt("device_rate_progress_per_min"),
Metadata: r.getInt("device_rate_metadata_per_min"),
}
}
// TombstoneTTL is how long deleted annotations are retained before purge.
func (r *SettingsRegistry) TombstoneTTL() time.Duration {
return time.Duration(r.getInt("annotation_tombstone_ttl_days")) * 24 * time.Hour
}
// ConversionCacheTTL is how long converted (kepub) files are served from cache.
func (r *SettingsRegistry) ConversionCacheTTL() time.Duration {
return time.Duration(r.getInt("conversion_cache_ttl_hours")) * time.Hour
}
// SyncQueueConfig bundles the sync queue interval and batch size. Read at
// startup; changes require a restart.
type SyncQueueConfig struct {
Interval time.Duration
BatchSize int
}
// SyncQueueConfig returns the active sync queue configuration.
func (r *SettingsRegistry) SyncQueueConfig() SyncQueueConfig {
return SyncQueueConfig{
Interval: time.Duration(r.getInt("sync_queue_interval_seconds")) * time.Second,
BatchSize: r.getInt("sync_queue_batch_size"),
}
}
// WorkerPoolConfig bundles worker count and queue capacity. Read at startup;
// changes require a restart.
type WorkerPoolConfig struct {
Size int
QueueCap int
}
// WorkerPoolConfig returns the active worker pool configuration.
func (r *SettingsRegistry) WorkerPoolConfig() WorkerPoolConfig {
return WorkerPoolConfig{
Size: r.getInt("worker_pool_size"),
QueueCap: r.getInt("worker_queue_cap"),
}
}
// SettingEntry exposes one setting's metadata + current value, for the admin UI/API.
type SettingEntry struct {
Key string `json:"key"`
Value string `json:"value"`
Type string `json:"type"`
Min string `json:"min,omitempty"`
Max string `json:"max,omitempty"`
RequiresRestart bool `json:"requires_restart"`
Category string `json:"category"`
Group string `json:"group"`
Description string `json:"description"`
IsDefault bool `json:"is_default"`
}
// All returns metadata + current values for every known setting, grouped by
// the in-memory cache (which reflects the DB after Load/Reload).
func (r *SettingsRegistry) All() []SettingEntry {
r.mu.RLock()
vals := make(map[string]string, len(r.values))
for k, v := range r.values {
vals[k] = v
}
r.mu.RUnlock()
out := make([]SettingEntry, 0, len(SettingDefaults))
for _, d := range SettingDefaults {
v, ok := vals[d.Key]
if !ok {
v = d.Value
}
out = append(out, SettingEntry{
Key: d.Key,
Value: v,
Type: d.Type,
Min: d.Min,
Max: d.Max,
RequiresRestart: d.RequiresRestart,
Category: d.Category,
Group: d.Group,
Description: d.Description,
IsDefault: v == d.Value,
})
}
return out
}
// LookupDefault returns the compiled-in SettingDefault for a key (ok=false if unknown).
func LookupDefault(key string) (SettingDefault, bool) {
d, ok := defaultBy[key]
return d, ok
}
@@ -0,0 +1,97 @@
package database
import (
"strconv"
"testing"
)
// TestSettingDefaults ensures every seeded setting has a compiled default with
// a valid value for its declared type. This guards against typos that would
// silently fall back at runtime.
func TestSettingDefaults(t *testing.T) {
if len(SettingDefaults) == 0 {
t.Fatal("SettingDefaults is empty")
}
for _, d := range SettingDefaults {
if d.Key == "" {
t.Errorf("default has empty key: %+v", d)
continue
}
switch d.Type {
case SettingTypeInt:
if _, err := strconv.Atoi(d.Value); err != nil {
t.Errorf("int setting %s default %q is not an int: %v", d.Key, d.Value, err)
}
if d.Min != "" {
if _, err := strconv.Atoi(d.Min); err != nil {
t.Errorf("int setting %s min %q is not an int", d.Key, d.Min)
}
}
if d.Max != "" {
if _, err := strconv.Atoi(d.Max); err != nil {
t.Errorf("int setting %s max %q is not an int", d.Key, d.Max)
}
}
case SettingTypeBool:
if _, err := strconv.ParseBool(d.Value); err != nil {
t.Errorf("bool setting %s default %q is not a bool", d.Key, d.Value)
}
case SettingTypeString:
if d.Value == "" {
t.Errorf("string setting %s has empty default", d.Key)
}
default:
t.Errorf("setting %s has unknown type %q", d.Key, d.Type)
}
}
}
// TestSettingsRegistryGetIntClamping verifies that out-of-range DB values are
// clamped to the declared min/max, and that garbage falls back to the default.
func TestSettingsRegistryGetIntClamping(t *testing.T) {
r := &SettingsRegistry{values: map[string]string{}, q: nil}
// Seed with an over-max value; expect clamping to the max (3600).
r.values["scan_poll_interval_seconds"] = "999999"
if got := r.ScanPollInterval(); got.Seconds() != 3600 {
t.Errorf("expected clamp to 3600, got %v", got)
}
// Seed with an under-min value; expect clamp to min (1).
r.values["scan_poll_interval_seconds"] = "0"
if got := r.ScanPollInterval(); got.Seconds() != 1 {
t.Errorf("expected clamp to 1, got %v", got)
}
// Seed with garbage; expect fallback to default (60).
r.values["scan_poll_interval_seconds"] = "not-a-number"
if got := r.ScanPollInterval(); got.Seconds() != 60 {
t.Errorf("expected fallback default 60, got %v", got)
}
}
// TestSettingsRegistryGetBoolFallback verifies bool parsing and fallback.
func TestSettingsRegistryGetBoolFallback(t *testing.T) {
r := &SettingsRegistry{values: map[string]string{}, q: nil}
r.values["auto_scan_enabled"] = "true"
if !r.AutoScanEnabled() {
t.Error("expected true")
}
r.values["auto_scan_enabled"] = "garbage"
// garbage falls back to default ("true")
if !r.AutoScanEnabled() {
t.Error("expected fallback to default true")
}
}
// TestLookupDefaultUnknownKey verifies unknown keys return ok=false.
func TestLookupDefaultUnknownKey(t *testing.T) {
if _, ok := LookupDefault("does_not_exist"); ok {
t.Error("expected ok=false for unknown key")
}
if _, ok := LookupDefault("session_duration_seconds"); !ok {
t.Error("expected ok=true for known key")
}
}
+167
View File
@@ -0,0 +1,167 @@
package handlers
import (
"context"
"net/http"
"time"
"bookhoard/internal/database"
wsync "bookhoard/internal/sync"
"github.com/google/uuid"
"github.com/jackc/pgx/v5/pgtype"
"github.com/labstack/echo/v5"
)
// DeletedAnnotationResponse is one entry of the deleted-annotation history
// for a book (the book page's "recently deleted" list). Restoring returns the
// row to the active set; purging removes it permanently.
type DeletedAnnotationResponse struct {
ID string `json:"id"`
AnnotationType string `json:"annotation_type"`
DisplayText string `json:"display_text"`
SecondaryText string `json:"secondary_text,omitempty"`
Color string `json:"color,omitempty"`
DeletedAt time.Time `json:"deleted_at"`
CreatedAt time.Time `json:"created_at"`
}
// DeletedAnnotationsForBook builds the deleted-annotation history for a user
// and book. Shared by the JSON API and the book page's server-rendered modal.
func DeletedAnnotationsForBook(ctx context.Context, db *database.Queries, userID, mediaItemID pgtype.UUID) []DeletedAnnotationResponse {
rows, err := db.ListDeletedAnnotationsForBook(ctx, database.ListDeletedAnnotationsForBookParams{
MediaItemID: mediaItemID,
UserID: userID,
})
if err != nil {
return []DeletedAnnotationResponse{}
}
response := make([]DeletedAnnotationResponse, 0, len(rows))
for _, row := range rows {
entry := DeletedAnnotationResponse{
ID: uuid.UUID(row.ID.Bytes).String(),
AnnotationType: row.AnnotationType,
DisplayText: row.DisplayText,
SecondaryText: row.SecondaryText.String,
Color: row.Color.String,
}
if row.DeletedAt.Valid {
entry.DeletedAt = row.DeletedAt.Time
}
if row.CreatedAt.Valid {
entry.CreatedAt = row.CreatedAt.Time
}
response = append(response, entry)
}
return response
}
// GetDeletedAnnotations handles GET /api/media-items/:id/annotations/deleted
func (mh *MediaHandler) GetDeletedAnnotations(c *echo.Context) error {
userUUID, mediaUUID, err := mh.parseUserAndMediaIDs(c)
if err != nil {
return err
}
response := DeletedAnnotationsForBook(c.Request().Context(), mh.db, userUUID, mediaUUID)
return c.JSON(http.StatusOK, map[string]interface{}{
"deleted_annotations": response,
"total": len(response),
})
}
// RestoreDeletedAnnotation handles POST /api/media-items/:id/annotations/:annotationId/restore
// Body/query: annotation_type=highlight|note|bookmark
func (mh *MediaHandler) RestoreDeletedAnnotation(c *echo.Context) error {
userUUID, mediaUUID, annotationUUID, kind, errResp := mh.parseAnnotationHistoryRequest(c, true)
if errResp != nil {
return errResp
}
if mh.annotationSvc == nil {
return c.JSON(http.StatusServiceUnavailable, map[string]string{"error": "annotation service unavailable"})
}
restored, err := mh.annotationSvc.RestoreAnnotationByID(c.Request().Context(), kind, userUUID, mediaUUID, annotationUUID)
if err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to restore annotation"})
}
if !restored {
return c.JSON(http.StatusNotFound, map[string]string{"error": "deleted annotation not found"})
}
return c.JSON(http.StatusOK, map[string]interface{}{"restored": true})
}
// PurgeDeletedAnnotation handles DELETE /api/media-items/:id/annotations/:annotationId
// Query: annotation_type=highlight|note|bookmark. Permanent — removes the
// tombstoned row from the history.
func (mh *MediaHandler) PurgeDeletedAnnotation(c *echo.Context) error {
userUUID, mediaUUID, annotationUUID, kind, errResp := mh.parseAnnotationHistoryRequest(c, false)
if errResp != nil {
return errResp
}
if mh.annotationSvc == nil {
return c.JSON(http.StatusServiceUnavailable, map[string]string{"error": "annotation service unavailable"})
}
purged, err := mh.annotationSvc.PurgeAnnotationByID(c.Request().Context(), kind, userUUID, mediaUUID, annotationUUID)
if err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to purge annotation"})
}
if !purged {
return c.JSON(http.StatusNotFound, map[string]string{"error": "deleted annotation not found"})
}
return c.JSON(http.StatusOK, map[string]interface{}{"purged": true})
}
// parseUserAndMediaIDs extracts the authenticated user and the media item
// from the route. A non-nil error has already been written as the response.
func (mh *MediaHandler) parseUserAndMediaIDs(c *echo.Context) (pgtype.UUID, pgtype.UUID, error) {
userID := c.Get("user_id").(string)
userUUID, err := uuid.Parse(userID)
if err != nil {
return pgtype.UUID{}, pgtype.UUID{}, c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid user"})
}
mediaID := c.Param("id")
mediaIDUUID, err := uuid.Parse(mediaID)
if err != nil {
return pgtype.UUID{}, pgtype.UUID{}, c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid media item id"})
}
return pgtype.UUID{Bytes: userUUID, Valid: true}, pgtype.UUID{Bytes: mediaIDUUID, Valid: true}, nil
}
// parseAnnotationHistoryRequest extracts user, media item, annotation ID, and
// the annotation_type (from query param or JSON body — restore posts a body,
// purge uses a query param). A non-nil error has already been written.
func (mh *MediaHandler) parseAnnotationHistoryRequest(c *echo.Context, allowBody bool) (pgtype.UUID, pgtype.UUID, pgtype.UUID, string, error) {
userUUID, mediaUUID, err := mh.parseUserAndMediaIDs(c)
if err != nil {
return pgtype.UUID{}, pgtype.UUID{}, pgtype.UUID{}, "", err
}
annotationID := c.Param("annotationId")
annotationUUID, err := uuid.Parse(annotationID)
if err != nil {
return pgtype.UUID{}, pgtype.UUID{}, pgtype.UUID{}, "", c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid annotation id"})
}
kind := c.QueryParam("annotation_type")
if kind == "" && allowBody {
var body struct {
AnnotationType string `json:"annotation_type"`
}
if c.Bind(&body) == nil {
kind = body.AnnotationType
}
}
if !wsync.ValidAnnotationKind(kind) {
return pgtype.UUID{}, pgtype.UUID{}, pgtype.UUID{}, "", c.JSON(http.StatusBadRequest, map[string]string{"error": "annotation_type must be highlight, note, or bookmark"})
}
return userUUID, mediaUUID, pgtype.UUID{Bytes: annotationUUID, Valid: true}, kind, nil
}
+32 -11
View File
@@ -26,14 +26,34 @@ import (
)
const (
// Session duration constants
// Follows same pattern as refresh_token.go
SessionDuration = 7 * 24 * time.Hour // 7 days
// DefaultSessionDuration is the fallback session duration used when no
// settings registry is wired (matches the historical 7-day value).
DefaultSessionDuration = 7 * 24 * time.Hour
)
// SessionDurationSec is the session duration in seconds for use in cookies and API responses
// Note: This is computed from SessionDuration to avoid magic numbers
var SessionDurationSec = int(SessionDuration.Seconds())
// SessionDurationSec is retained for backward compatibility; new code uses the
// registry via AuthHandler.sessionDuration().
var SessionDurationSec = int(DefaultSessionDuration.Seconds())
// SetSettings wires the tunable settings registry (optional).
func (h *AuthHandler) SetSettings(s *database.SettingsRegistry) { h.settings = s }
// sessionDuration returns the active session duration from the registry.
func (h *AuthHandler) sessionDuration() time.Duration {
if h.settings != nil {
return h.settings.SessionDuration()
}
return DefaultSessionDuration
}
// refreshTokenTTL returns the active refresh-token lifetime (shared with the
// session duration), with a compiled-default fallback.
func (h *AuthHandler) refreshTokenTTL() time.Duration {
if h.settings != nil {
return h.settings.SessionDuration()
}
return DefaultSessionDuration
}
var secure = os.Getenv("COOKIE_SECURE")
@@ -41,6 +61,7 @@ type AuthHandler struct {
db *database.Queries
jwtKey []byte
loginAttemptTracker *middleware.LoginAttemptTracker
settings *database.SettingsRegistry
}
func NewAuthHandler(db *database.Queries, jwtSecret string, loginAttemptTracker *middleware.LoginAttemptTracker) *AuthHandler {
@@ -265,7 +286,7 @@ func (h *AuthHandler) Register(c *echo.Context) error {
HttpOnly: true,
Secure: secure == "true", // TODO: Set to true in production with HTTPS
SameSite: http.SameSiteLaxMode,
MaxAge: SessionDurationSec,
MaxAge: int(h.sessionDuration().Seconds()),
}
c.SetCookie(cookie)
@@ -298,7 +319,7 @@ window.location.href = '/dashboard';
Token: accessToken,
RefreshToken: refreshToken,
TokenType: "Bearer",
ExpiresIn: SessionDurationSec,
ExpiresIn: int(h.sessionDuration().Seconds()),
User: UserProfile{
ID: uuid.UUID(user.ID.Bytes).String(),
Email: user.Email,
@@ -411,7 +432,7 @@ func (h *AuthHandler) Login(c *echo.Context) error {
HttpOnly: true,
Secure: secure == "true", // TODO: Set to true in production with HTTPS
SameSite: http.SameSiteLaxMode,
MaxAge: SessionDurationSec,
MaxAge: int(h.sessionDuration().Seconds()),
}
c.SetCookie(cookie)
@@ -450,7 +471,7 @@ window.location.href = '%s';
Token: accessToken,
RefreshToken: refreshToken,
TokenType: "Bearer",
ExpiresIn: SessionDurationSec,
ExpiresIn: int(h.sessionDuration().Seconds()),
User: UserProfile{
ID: uuid.UUID(user.ID.Bytes).String(),
Email: user.Email,
@@ -1014,7 +1035,7 @@ func (h *AuthHandler) generateJWTWithAllClaims(userID, userRole, userEmail, user
"user_role": userRole,
"user_email": userEmail,
"user_username": userUsername,
"exp": time.Now().Add(SessionDuration).Unix(),
"exp": time.Now().Add(h.sessionDuration()).Unix(),
"iat": time.Now().Unix(),
}
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
+1
View File
@@ -73,6 +73,7 @@ type BookInfo struct {
Title string `json:"title"`
Author string `json:"author"`
CoverImagePath string `json:"cover_image_path"`
HasConflict bool `json:"has_conflict"`
}
type SectionData struct {
+55
View File
@@ -4,6 +4,7 @@ import (
"bookhoard/internal/database"
"bookhoard/internal/services"
"bookhoard/internal/utils"
"context"
"log"
"net/http"
"strconv"
@@ -64,6 +65,7 @@ func (h *DashboardHandler) GetSections(c *echo.Context) error {
}
sectionData := BuildSections(sections, libraryID)
sectionData = MarkActiveConflictsSections(c.Request().Context(), h.db, user.ID, sectionData)
return c.JSON(http.StatusOK, map[string]interface{}{"sections": sectionData})
}
@@ -188,6 +190,59 @@ func BuildSections(sections []services.DashboardSection, currentLibraryID string
return result
}
// activeConflictSet returns the set of media item IDs (as strings) that have an
// active (unresolved) progress sync conflict for the given user. A single query
// is issued; resolved conflicts are filtered out in memory.
func activeConflictSet(ctx context.Context, db *database.Queries, userID pgtype.UUID) map[string]bool {
conflicts, err := db.ListSyncConflictsByUser(ctx, userID)
if err != nil {
return nil
}
set := make(map[string]bool, len(conflicts))
for _, c := range conflicts {
if c.ResolutionStatus.String == "unresolved" {
set[uuid.UUID(c.MediaItemID.Bytes).String()] = true
}
}
return set
}
// MarkActiveConflicts stamps HasConflict on each book whose media item has an
// active progress sync conflict for the user. It performs a single query
// regardless of how many books are passed.
func MarkActiveConflicts(ctx context.Context, db *database.Queries, userID pgtype.UUID, books []BookInfo) []BookInfo {
if len(books) == 0 {
return books
}
set := activeConflictSet(ctx, db, userID)
for i := range books {
if set[books[i].MediaItemID] {
books[i].HasConflict = true
}
}
return books
}
// MarkActiveConflictsSections is the section-aware variant of MarkActiveConflicts,
// used by the dashboard which renders books grouped into sections.
func MarkActiveConflictsSections(ctx context.Context, db *database.Queries, userID pgtype.UUID, sections []SectionData) []SectionData {
if len(sections) == 0 {
return sections
}
set := activeConflictSet(ctx, db, userID)
if len(set) == 0 {
return sections
}
for s := range sections {
for i := range sections[s].Items {
if set[sections[s].Items[i].MediaItemID] {
sections[s].Items[i].HasConflict = true
}
}
}
return sections
}
func getViewAllURL(collectionID string, libraryID string) string {
if collectionID != "" {
if libraryID != "" {
+255
View File
@@ -0,0 +1,255 @@
package handlers
import (
"bookhoard/internal/database"
"fmt"
"net/http"
"time"
"github.com/google/uuid"
"github.com/jackc/pgx/v5/pgtype"
"github.com/labstack/echo/v5"
)
// HashConflictsHandler serves the admin Hash Conflicts page API: listing
// content-duplicate groups (same library + SHA-256 at different paths) and
// resolving them by keeping every copy or merging all but one.
type HashConflictsHandler struct {
db *database.Queries
}
func NewHashConflictsHandler(db *database.Queries) *HashConflictsHandler {
return &HashConflictsHandler{db: db}
}
// HashConflictItem is one copy in a conflict group, hydrated with per-item
// user-data counts so the admin can make an informed keep/merge choice.
type HashConflictItem struct {
ID uuid.UUID `json:"id"`
Title string `json:"title"`
Author string `json:"author,omitempty"`
FilePath string `json:"file_path"`
FileSize int64 `json:"file_size,omitempty"`
CreatedAt string `json:"created_at"`
ProgressCount int64 `json:"progress_count"`
HighlightCount int64 `json:"highlight_count"`
BookmarkCount int64 `json:"bookmark_count"`
NoteCount int64 `json:"note_count"`
CollectionCount int64 `json:"collection_count"`
}
// HashConflictResponse is one pending conflict group.
type HashConflictResponse struct {
ID string `json:"id"`
LibraryID string `json:"library_id"`
LibraryName string `json:"library_name"`
SHA256 string `json:"sha256"`
CreatedAt string `json:"created_at"`
Items []HashConflictItem `json:"items"`
}
// ListHashConflicts returns all pending hash conflict groups with their member
// items and usage counts.
// GET /api/admin/hash-conflicts
func (h *HashConflictsHandler) ListHashConflicts(c *echo.Context) error {
ctx := c.Request().Context()
pending, err := h.db.ListPendingHashConflicts(ctx)
if err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{
"error": "failed to list hash conflicts",
})
}
conflicts := make([]HashConflictResponse, 0, len(pending))
for _, p := range pending {
resp := HashConflictResponse{
ID: uuid.UUID(p.ID.Bytes).String(),
LibraryID: uuid.UUID(p.LibraryID.Bytes).String(),
LibraryName: p.LibraryName,
SHA256: p.FileSha256,
CreatedAt: p.CreatedAt.Time.Format(time.RFC3339),
Items: []HashConflictItem{},
}
items, err := h.db.ListMediaItemsBySHA256AndLibrary(ctx, database.ListMediaItemsBySHA256AndLibraryParams{
FileSha256: pgtype.Text{String: p.FileSha256, Valid: true},
LibraryID: p.LibraryID,
})
if err != nil {
continue
}
for _, mi := range items {
counts, err := h.db.GetMediaItemUsageCounts(ctx, mi.ID)
if err != nil {
counts = database.GetMediaItemUsageCountsRow{}
}
resp.Items = append(resp.Items, HashConflictItem{
ID: uuid.UUID(mi.ID.Bytes),
Title: mi.Title,
Author: mi.Author.String,
FilePath: mi.FilePath,
FileSize: mi.FileSize.Int64,
CreatedAt: mi.CreatedAt.Time.Format(time.RFC3339),
ProgressCount: counts.ProgressCount,
HighlightCount: counts.HighlightsCount,
BookmarkCount: counts.BookmarksCount,
NoteCount: counts.NotesCount,
CollectionCount: counts.CollectionsCount,
})
}
conflicts = append(conflicts, resp)
}
return c.JSON(http.StatusOK, map[string]interface{}{
"conflicts": conflicts,
"total": len(conflicts),
})
}
// ResolveHashConflict resolves one conflict group.
//
// Form/JSON fields:
// - action=keep_all both copies are intentional; dismiss
// - action=keep&keep_uuid=<uuid> merge every other copy's child rows into the
// kept item (progress, highlights, bookmarks,
// notes, collections, ...) and delete the losers
//
// POST /api/admin/hash-conflicts/:id/resolve
func (h *HashConflictsHandler) ResolveHashConflict(c *echo.Context) error {
ctx := c.Request().Context()
conflictID, err := uuid.Parse(c.Param("id"))
if err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid conflict ID"})
}
pgConflictID := pgtype.UUID{Bytes: conflictID, Valid: true}
conflict, err := h.db.GetHashConflict(ctx, pgConflictID)
if err != nil {
return c.JSON(http.StatusNotFound, map[string]string{"error": "conflict not found"})
}
if conflict.Status != "pending" {
return c.JSON(http.StatusConflict, map[string]string{"error": "conflict already resolved"})
}
action := c.FormValue("action")
keepUUIDStr := c.FormValue("keep_uuid")
if action == "" {
// Also accept a JSON body (htmx sends form-encoded, API clients may send JSON)
var body struct {
Action string `json:"action"`
KeepUUID string `json:"keep_uuid"`
}
if err := c.Bind(&body); err == nil && body.Action != "" {
action = body.Action
if keepUUIDStr == "" {
keepUUIDStr = body.KeepUUID
}
}
}
var pgUserID pgtype.UUID
if userID, ok := c.Get("user_id").(string); ok && userID != "" {
if u, err := uuid.Parse(userID); err == nil {
pgUserID = pgtype.UUID{Bytes: u, Valid: true}
}
}
switch action {
case "keep_all":
if err := h.db.ResolveHashConflict(ctx, database.ResolveHashConflictParams{
ID: pgConflictID,
Resolution: pgtype.Text{String: "keep_all", Valid: true},
ResolvedBy: pgUserID,
}); err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to resolve conflict"})
}
return renderResolved(c, "All copies kept.")
case "keep":
keepUUID, err := uuid.Parse(keepUUIDStr)
if err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "keep_uuid is required for action=keep"})
}
pgKeepUUID := pgtype.UUID{Bytes: keepUUID, Valid: true}
// Validate the kept item belongs to this conflict group.
items, err := h.db.ListMediaItemsBySHA256AndLibrary(ctx, database.ListMediaItemsBySHA256AndLibraryParams{
FileSha256: pgtype.Text{String: conflict.FileSha256, Valid: true},
LibraryID: conflict.LibraryID,
})
if err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to load conflict group"})
}
keepValid := false
for _, mi := range items {
if mi.ID.Bytes == pgKeepUUID.Bytes {
keepValid = true
break
}
}
if !keepValid {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "keep_uuid is not part of this conflict"})
}
merged := 0
for _, mi := range items {
if mi.ID.Bytes == pgKeepUUID.Bytes {
continue
}
if err := h.db.ReparentMediaItemChildren(ctx, database.ReparentMediaItemChildrenParams{
Column1: pgKeepUUID,
Column2: mi.ID,
}); err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{
"error": fmt.Sprintf("failed to merge %q: %v", mi.FilePath, err),
})
}
if err := h.db.DeleteMediaItem(ctx, mi.ID); err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{
"error": fmt.Sprintf("failed to delete %q: %v", mi.FilePath, err),
})
}
merged++
}
if err := h.db.ResolveHashConflict(ctx, database.ResolveHashConflictParams{
ID: pgConflictID,
Resolution: pgtype.Text{String: "kept:" + keepUUID.String(), Valid: true},
ResolvedBy: pgUserID,
}); err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to resolve conflict"})
}
return renderResolved(c, fmt.Sprintf("Merged %d duplicate cop%s - all reading data preserved.",
merged, map[bool]string{true: "y", false: "ies"}[merged == 1]))
default:
return c.JSON(http.StatusBadRequest, map[string]string{"error": "action must be 'keep_all' or 'keep'"})
}
}
// renderResolved returns the htmx fragment swapped in place of a conflict card.
// Built inline (rather than via the templates package) because templates
// imports handlers and a back-import would be a cycle.
func renderResolved(c *echo.Context, message string) error {
html := fmt.Sprintf(`
<div class="card p-6 flex items-center gap-3">
<span class="grid place-items-center h-10 w-10 rounded-xl shrink-0"
style="background-color: var(--accent-muted); color: var(--accent);">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="h-5 w-5" aria-hidden="true">
<path d="M22 11.08V12a10 10 0 1 1-5.93-9.14"></path>
<polyline points="22 4 12 14.01 9 11.01"></polyline>
</svg>
</span>
<div>
<p class="font-medium" style="color: var(--text-primary);">Conflict resolved</p>
<p class="text-sm" style="color: var(--text-secondary);">%s</p>
</div>
</div>`, message)
return c.HTML(http.StatusOK, html)
}
+7 -4
View File
@@ -2,6 +2,7 @@ package handlers
import (
"bookhoard/internal/database"
"bookhoard/internal/services"
wsync "bookhoard/internal/sync"
"encoding/json"
"fmt"
@@ -22,10 +23,11 @@ type KoboHandler struct {
progressSvc *wsync.ProgressService
annotationSvc *wsync.AnnotationService
libraryService LibraryPathResolver
bookResolver *services.BookResolver
}
func NewKoboHandler(db *database.Queries, connManager *wsync.ConnectionManager) *KoboHandler {
return &KoboHandler{db: db, connManager: connManager}
return &KoboHandler{db: db, connManager: connManager, bookResolver: services.NewBookResolver(db)}
}
func (h *KoboHandler) SetProgressService(svc *wsync.ProgressService) {
@@ -52,8 +54,9 @@ func (h *KoboHandler) mapContentIdToBookhoardUUID(ctx *echo.Context, contentId s
// Step 2: ContentId not found - check if it looks like a SHA-256 hash
if len(contentId) == 64 && looksLikeSHA256(contentId) {
// Try to find media item by SHA-256
mediaItem, err := h.db.GetMediaItemBySHA256(ctx.Request().Context(), pgtype.Text{String: contentId, Valid: true})
// Try to find media item by SHA-256 (format-aware: also checks
// media_item_formats, so a converted/alternate format hash matches).
mediaItem, _, err := h.bookResolver.ResolveBySHA256(ctx.Request().Context(), contentId)
if err == nil {
// Found by SHA-256! Create device catalog entry for future lookups
_, _ = h.db.CreateDeviceCatalog(ctx.Request().Context(), database.CreateDeviceCatalogParams{
@@ -623,7 +626,7 @@ func (h *KoboHandler) Markup(c *echo.Context) error {
}
if h.annotationSvc != nil && len(processedBooks) > 0 {
cutoff := pgtype.Timestamptz{Time: time.Now().Add(-wsync.TombstoneTTL), Valid: true}
cutoff := pgtype.Timestamptz{Time: time.Now().Add(-h.annotationSvc.ActiveTombstoneTTL()), Valid: true}
for mediaItemID, contentId := range processedBooks {
tombstones, _ := h.db.GetTombstonedAnnotationsForBook(c.Request().Context(), database.GetTombstonedAnnotationsForBookParams{
MediaItemID: mediaItemID,
+500 -100
View File
@@ -2,13 +2,17 @@ package handlers
import (
"bookhoard/internal/database"
"bookhoard/internal/services"
wsync "bookhoard/internal/sync"
"context"
"encoding/json"
"fmt"
"log"
"net/http"
"strconv"
"strings"
"time"
"unicode/utf8"
"github.com/google/uuid"
"github.com/jackc/pgx/v5/pgtype"
@@ -22,6 +26,7 @@ type KOReaderHandler struct {
progressSvc *wsync.ProgressService
annotationSvc *wsync.AnnotationService
libraryService LibraryPathResolver
bookResolver *services.BookResolver
}
type LibraryPathResolver interface {
@@ -29,7 +34,12 @@ type LibraryPathResolver interface {
}
func NewKOReaderHandler(db *database.Queries, connManager *wsync.ConnectionManager, queue *wsync.SyncQueueProcessor) *KOReaderHandler {
return &KOReaderHandler{db: db, connManager: connManager, queue: queue}
return &KOReaderHandler{
db: db,
connManager: connManager,
queue: queue,
bookResolver: services.NewBookResolver(db),
}
}
func (h *KOReaderHandler) SetProgressService(svc *wsync.ProgressService) {
@@ -40,7 +50,7 @@ func (h *KOReaderHandler) SetAnnotationService(svc *wsync.AnnotationService) {
h.annotationSvc = svc
}
func (h *KOReaderHandler) convertHighlightPositions(ctx context.Context, mediaItemID pgtype.UUID, pos0, pos1 string) (string, string) {
func (h *KOReaderHandler) convertHighlightPositions(ctx context.Context, mediaItemID pgtype.UUID, pos0, pos1, contextText string) (string, string) {
if pos0 == "" || h.libraryService == nil {
return "", ""
}
@@ -52,9 +62,89 @@ func (h *KOReaderHandler) convertHighlightPositions(ctx context.Context, mediaIt
if err != nil || epubPath == "" {
return "", ""
}
startLoc := wsync.ConvertToCanonical(wsync.LocatorSourceKOReader, pos0, 0, "", mediaItem.FormatGroup, epubPath, "")
// The annotation's own text is the ideal anchor for the converter's
// text-search path: clients (thin, underpowered) send only raw
// locators, the server resolves them against the actual book.
startLoc := wsync.ConvertToCanonical(wsync.LocatorSourceKOReader, pos0, 0, contextText, mediaItem.FormatGroup, epubPath, "")
endLoc := wsync.ConvertToCanonical(wsync.LocatorSourceKOReader, pos1, 0, "", mediaItem.FormatGroup, epubPath, "")
return startLoc.CFI, endLoc.CFI
endCFI := endLoc.CFI
// The end conversion carries no context text, so unless it resolved
// exactly it degenerates to a percentage fallback anchored at the
// document start — useless as a range end. When the START resolved
// exactly, derive the end from it: same node, character offset
// advanced by the selection's UTF-16 length (the CFI offset unit).
if endLoc.Precision != "exact" && startLoc.Precision == "exact" && contextText != "" {
endCFI = extendCFIByLength(startLoc.CFI, contextText)
}
return startLoc.CFI, endCFI
}
// extendCFIByLength advances a point CFI's trailing character offset by the
// UTF-16 length of text (EPUB CFI character offsets are UTF-16 code units).
// Selections spanning multiple text nodes produce an out-of-range offset —
// harmless: resolution clamps or fails, and consumers fall back to the start.
func extendCFIByLength(cfi, text string) string {
if cfi == "" || text == "" {
return cfi
}
i := strings.LastIndex(cfi, ":")
if i < 0 || !strings.HasSuffix(cfi, ")") {
return cfi
}
off, err := strconv.Atoi(cfi[i+1 : len(cfi)-1])
if err != nil {
return cfi
}
utf16len := 0
for _, r := range text {
if r > 0xFFFF {
utf16len += 2
} else {
utf16len++
}
}
return cfi[:i+1] + strconv.Itoa(off+utf16len) + ")"
}
// existingHighlightColor returns the stored color of the highlight matching
// the dedup key ("" when none) so device echoes that carry no color never
// clobber the web color.
func (h *KOReaderHandler) existingHighlightColor(ctx context.Context, mediaItemID, userID pgtype.UUID, dedupKey string) string {
if dedupKey == "" {
return ""
}
existing, err := h.db.GetMediaHighlightByDedupKey(ctx, database.GetMediaHighlightByDedupKeyParams{
UserID: userID,
MediaItemID: mediaItemID,
DedupKey: pgtype.Text{String: dedupKey, Valid: true},
})
if err != nil {
return ""
}
return existing.Color.String
}
// deriveAnnotationPercentage computes a percentage for device-pushed
// annotations when the client didn't send one (thin clients skip their own
// per-annotation page lookups; arithmetic is only free on paging documents).
func (h *KOReaderHandler) deriveAnnotationPercentage(ctx context.Context, mediaItemID pgtype.UUID, pos0 string, page int) float64 {
mediaItem, err := h.db.GetMediaItem(ctx, mediaItemID)
if err != nil {
return 0
}
formatGroup := wsync.FormatGroup(mediaItem.FormatGroup)
if formatGroup == wsync.FormatGroupFixedLayout || formatGroup == wsync.FormatGroupComicArchive {
if page > 0 && mediaItem.PageCount.Valid && mediaItem.PageCount.Int32 > 0 {
return float64(page) / float64(mediaItem.PageCount.Int32)
}
return 0
}
if wsync.IsCREXPointer(pos0) && h.libraryService != nil {
if epubPath, err := h.libraryService.ResolveMediaPath(ctx, mediaItem.LibraryID, mediaItem.FilePath); err == nil && epubPath != "" {
return wsync.NewCFIConverter(epubPath).SectionPercentage(pos0)
}
}
return 0
}
func (h *KOReaderHandler) SetLibraryService(svc LibraryPathResolver) {
@@ -68,24 +158,35 @@ type KOReaderProgressRequest struct {
}
type KOReaderBookProgress struct {
UUID string `json:"uuid,omitempty"`
SHA256 string `json:"sha256,omitempty"`
Title string `json:"title,omitempty"`
Authors []string `json:"authors,omitempty"`
Progress float64 `json:"progress"`
Percentage float64 `json:"percentage"`
LastRead string `json:"last_read,omitempty"`
FilePath string `json:"file_path,omitempty"`
DeviceInfo KOReaderDeviceInfo `json:"device_info,omitempty"`
Bookmarks []KOReaderBookmark `json:"bookmarks,omitempty"`
Highlights []KOReaderHighlight `json:"highlights,omitempty"`
Notes []KOReaderNote `json:"notes,omitempty"`
Chapter *int `json:"chapter,omitempty"`
Character *int64 `json:"character,omitempty"`
Epubcfi *string `json:"epubcfi,omitempty"`
ContextText *string `json:"context_text,omitempty"`
Page *int `json:"page,omitempty"`
TotalPages *int `json:"total_pages,omitempty"`
UUID string `json:"uuid,omitempty"`
SHA256 string `json:"sha256,omitempty"`
Title string `json:"title,omitempty"`
Authors []string `json:"authors,omitempty"`
Progress float64 `json:"progress"`
Percentage float64 `json:"percentage"`
LastRead string `json:"last_read,omitempty"`
FilePath string `json:"file_path,omitempty"`
DeviceInfo KOReaderDeviceInfo `json:"device_info,omitempty"`
Bookmarks []KOReaderBookmark `json:"bookmarks,omitempty"`
Highlights []KOReaderHighlight `json:"highlights,omitempty"`
Notes []KOReaderNote `json:"notes,omitempty"`
Chapter *int `json:"chapter,omitempty"`
Character *int64 `json:"character,omitempty"`
Epubcfi *string `json:"epubcfi,omitempty"`
ContextText *string `json:"context_text,omitempty"`
Page *int `json:"page,omitempty"`
TotalPages *int `json:"total_pages,omitempty"`
// Device-side deletions, reported by dedup key. Keys refer to annotations
// the device previously received from the server (or echoes of its own
// pushes); the device only flags a deletion after observing the key in a
// pull, so absence from these arrays is never interpreted as deletion.
DeletedHighlights []KOReaderDeletedAnnotation `json:"deleted_highlights,omitempty"`
DeletedBookmarks []KOReaderDeletedAnnotation `json:"deleted_bookmarks,omitempty"`
}
// KOReaderDeletedAnnotation identifies a deleted annotation by dedup key.
type KOReaderDeletedAnnotation struct {
DedupKey string `json:"dedup_key"`
}
type KOReaderDeviceInfo struct {
@@ -93,53 +194,91 @@ type KOReaderDeviceInfo struct {
DeviceModel string `json:"device_model,omitempty"`
}
// FlexInt tolerates the loose types KOReader clients send for optional
// numeric fields: JSON numbers, numeric strings ("30"), empty strings
// (""), or non-numeric strings ("/body/..." xpointers in `page` for CRE
// documents) — the latter decode to 0. Without this, a single annotation
// carrying chapter:"" or page:"/body/..." failed the whole request bind
// with a 400.
type FlexInt int
func (f *FlexInt) UnmarshalJSON(b []byte) error {
s := strings.TrimSpace(string(b))
if s == "null" || s == `""` {
*f = 0
return nil
}
if n, err := strconv.Atoi(s); err == nil {
*f = FlexInt(n)
return nil
}
if strings.HasPrefix(s, `"`) && strings.HasSuffix(s, `"`) {
inner := s[1 : len(s)-1]
if n, err := strconv.Atoi(inner); err == nil {
*f = FlexInt(n)
return nil
}
*f = 0
return nil
}
if fl, err := strconv.ParseFloat(s, 64); err == nil {
*f = FlexInt(int(fl))
return nil
}
*f = 0
return nil
}
type KOReaderBookmark struct {
Chapter int `json:"chapter,omitempty"`
Chapter FlexInt `json:"chapter,omitempty"`
Datetime string `json:"datetime,omitempty"`
Notes string `json:"notes,omitempty"`
Pos0 string `json:"pos0,omitempty"`
Pos1 string `json:"pos1,omitempty"`
Page int `json:"page,omitempty"`
Page FlexInt `json:"page,omitempty"`
Text string `json:"text,omitempty"`
Type string `json:"type,omitempty"`
Percentage *float64 `json:"percentage,omitempty"`
BookSHA256 string `json:"book_sha256,omitempty"`
DedupKey string `json:"dedup_key,omitempty"`
}
type KOReaderHighlight struct {
Chapter int `json:"chapter,omitempty"`
Chapter FlexInt `json:"chapter,omitempty"`
Datetime string `json:"datetime,omitempty"`
Notes string `json:"notes,omitempty"`
Pos0 string `json:"pos0,omitempty"`
Pos1 string `json:"pos1,omitempty"`
Page int `json:"page,omitempty"`
Page FlexInt `json:"page,omitempty"`
Text string `json:"text,omitempty"`
Type string `json:"type,omitempty"`
Color string `json:"color,omitempty"`
Percentage *float64 `json:"percentage,omitempty"`
BookSHA256 string `json:"book_sha256,omitempty"`
DedupKey string `json:"dedup_key,omitempty"`
}
type KOReaderNote struct {
Chapter int `json:"chapter,omitempty"`
Chapter FlexInt `json:"chapter,omitempty"`
Datetime string `json:"datetime,omitempty"`
Notes string `json:"notes,omitempty"`
Pos0 string `json:"pos0,omitempty"`
Pos1 string `json:"pos1,omitempty"`
Page int `json:"page,omitempty"`
Page FlexInt `json:"page,omitempty"`
Text string `json:"text,omitempty"`
Type string `json:"type,omitempty"`
Percentage *float64 `json:"percentage,omitempty"`
BookSHA256 string `json:"book_sha256,omitempty"`
DedupKey string `json:"dedup_key,omitempty"`
}
type KOReaderSyncResponse struct {
SyncStatus string `json:"sync_status"`
BooksSynced int `json:"books_synced"`
SyncStatus string `json:"sync_status"`
BooksSynced int `json:"books_synced"`
BookResults []KOReaderBookSyncResult `json:"book_results,omitempty"`
Conflicts []KOReaderConflict `json:"conflicts,omitempty"`
Timestamp string `json:"timestamp"`
DeviceUpdated bool `json:"device_updated"`
Conflicts []KOReaderConflict `json:"conflicts,omitempty"`
Timestamp string `json:"timestamp"`
DeviceUpdated bool `json:"device_updated"`
}
type KOReaderBookSyncResult struct {
@@ -158,6 +297,7 @@ type KOReaderConflict struct {
type KOReaderMetadata struct {
UUID string `json:"uuid"`
SHA256 string `json:"sha256,omitempty"`
Title string `json:"title"`
Authors []string `json:"authors"`
Progress KOReaderProgressData `json:"progress"`
@@ -166,20 +306,20 @@ type KOReaderMetadata struct {
}
type KOReaderProgressData struct {
Percentage float64 `json:"percentage"`
Character *int64 `json:"character,omitempty"`
Epubcfi *string `json:"epubcfi,omitempty"`
KoreaderXPointer *string `json:"koreader_xpointer,omitempty"`
Chapter *int `json:"chapter,omitempty"`
ChapterProgress *float64 `json:"chapter_progress,omitempty"`
Page *int `json:"page,omitempty"`
TotalPages *int `json:"total_pages,omitempty"`
Percentage float64 `json:"percentage"`
Character *int64 `json:"character,omitempty"`
Epubcfi *string `json:"epubcfi,omitempty"`
KoreaderXPointer *string `json:"koreader_xpointer,omitempty"`
Chapter *int `json:"chapter,omitempty"`
ChapterProgress *float64 `json:"chapter_progress,omitempty"`
Page *int `json:"page,omitempty"`
TotalPages *int `json:"total_pages,omitempty"`
}
type KOReaderAnnotations struct {
Highlights []KOReaderHighlight `json:"highlights,omitempty"`
Notes []KOReaderNote `json:"notes,omitempty"`
Bookmarks []KOReaderBookmark `json:"bookmarks,omitempty"`
Highlights []KOReaderHighlight `json:"highlights,omitempty"`
Notes []KOReaderNote `json:"notes,omitempty"`
Bookmarks []KOReaderBookmark `json:"bookmarks,omitempty"`
DeletedHighlights []map[string]interface{} `json:"deleted_highlights,omitempty"`
DeletedBookmarks []map[string]interface{} `json:"deleted_bookmarks,omitempty"`
}
@@ -192,6 +332,7 @@ type KOReaderLibraryResponse struct {
type KOReaderLibraryBook struct {
UUID string `json:"uuid"`
SHA256 string `json:"sha256,omitempty"`
Title string `json:"title"`
Author string `json:"author"`
ContentType string `json:"content_type"`
@@ -303,8 +444,10 @@ func (h *KOReaderHandler) resolveBookToMediaItem(c *echo.Context, deviceID pgtyp
}
// Priority 2: SHA-256 provided (medium confidence - 0.9)
// Uses the shared BookResolver, which also checks per-format hashes
// (media_item_formats) so a converted file (KEPUB/PDF) matches too.
if book.SHA256 != "" && len(book.SHA256) == 64 {
mediaItem, err := h.db.GetMediaItemBySHA256(ctx, pgtype.Text{String: book.SHA256, Valid: true})
mediaItem, _, err := h.bookResolver.ResolveBySHA256(ctx, book.SHA256)
if err == nil {
// Create device file alias if FilePath is provided
if book.FilePath != "" {
@@ -475,12 +618,16 @@ func (h *KOReaderHandler) processBookAnnotations(ctx context.Context, deviceID,
for _, hl := range book.Highlights {
startPos := hl.Pos0
endPos := hl.Pos1
epubcfiStart, epubcfiEnd := h.convertHighlightPositions(ctx, mediaItemID, startPos, endPos)
// The highlight's own text anchors the conversion exactly.
epubcfiStart, epubcfiEnd := h.convertHighlightPositions(ctx, mediaItemID, startPos, endPos, hl.Text)
pctStart := 0.0
if hl.Percentage != nil {
pctStart = *hl.Percentage
}
if pctStart == 0 {
pctStart = h.deriveAnnotationPercentage(ctx, mediaItemID, startPos, int(hl.Page))
}
deviceData, _ := json.Marshal(map[string]interface{}{
"datetime": hl.Datetime,
@@ -489,31 +636,55 @@ func (h *KOReaderHandler) processBookAnnotations(ctx context.Context, deviceID,
"page": hl.Page,
})
// Color semantics: devices render their own default and cannot
// round-trip web colors. An echo carries NO color — preserve the
// stored (web) color so round-trips never change it. A non-empty
// color means the user edited the highlight on the device: map the
// device color name and let it win.
color := ""
if hl.Color != "" {
color = mapColorFromKOReader(hl.Color)
}
dedupKey := hl.DedupKey
if dedupKey == "" {
dedupKey = wsync.ComputeDedupKey(hl.Text, epubcfiStart, startPos)
}
if color == "" {
color = h.existingHighlightColor(ctx, mediaItemID, userID, dedupKey)
}
if color == "" {
color = "#ffd54f"
}
h.annotationSvc.SaveHighlight(ctx, wsync.SaveHighlightRequest{
MediaItemID: mediaItemID,
UserID: userID,
SelectionText: hl.Text,
StartPosition: startPos,
EndPosition: endPos,
Color: hl.Color,
NoteText: hl.Notes,
PercentageStart: pctStart,
EpubcfiStart: epubcfiStart,
EpubcfiEnd: epubcfiEnd,
Source: "koreader",
DeviceSyncData: deviceData,
MediaItemID: mediaItemID,
UserID: userID,
SelectionText: hl.Text,
StartPosition: startPos,
EndPosition: endPos,
Color: color,
NoteText: hl.Notes,
PercentageStart: pctStart,
EpubcfiStart: epubcfiStart,
EpubcfiEnd: epubcfiEnd,
Source: "koreader",
DeviceSyncData: deviceData,
DedupKey: dedupKey,
})
}
for _, note := range book.Notes {
startPos := note.Pos0
endPos := note.Pos1
epubcfiStart, epubcfiEnd := h.convertHighlightPositions(ctx, mediaItemID, startPos, endPos)
epubcfiStart, epubcfiEnd := h.convertHighlightPositions(ctx, mediaItemID, startPos, endPos, note.Text)
pctStart := 0.0
if note.Percentage != nil {
pctStart = *note.Percentage
}
if pctStart == 0 {
pctStart = h.deriveAnnotationPercentage(ctx, mediaItemID, startPos, int(note.Page))
}
deviceData, _ := json.Marshal(map[string]interface{}{
"datetime": note.Datetime,
@@ -522,18 +693,25 @@ func (h *KOReaderHandler) processBookAnnotations(ctx context.Context, deviceID,
"page": note.Page,
})
dedupKey := note.DedupKey
if dedupKey == "" {
dedupKey = wsync.ComputeDedupKey(note.Text, epubcfiStart, startPos)
}
h.annotationSvc.SaveHighlight(ctx, wsync.SaveHighlightRequest{
MediaItemID: mediaItemID,
UserID: userID,
SelectionText: note.Text,
StartPosition: startPos,
EndPosition: endPos,
NoteText: note.Notes,
PercentageStart: pctStart,
EpubcfiStart: epubcfiStart,
EpubcfiEnd: epubcfiEnd,
Source: "koreader",
DeviceSyncData: deviceData,
MediaItemID: mediaItemID,
UserID: userID,
SelectionText: note.Text,
StartPosition: startPos,
EndPosition: endPos,
Color: h.existingHighlightColor(ctx, mediaItemID, userID, dedupKey),
NoteText: note.Notes,
PercentageStart: pctStart,
EpubcfiStart: epubcfiStart,
EpubcfiEnd: epubcfiEnd,
Source: "koreader",
DeviceSyncData: deviceData,
DedupKey: dedupKey,
})
}
@@ -551,6 +729,11 @@ func (h *KOReaderHandler) processBookAnnotations(ctx context.Context, deviceID,
"page": bookmark.Page,
})
dedupKey := bookmark.DedupKey
if dedupKey == "" {
dedupKey = wsync.ComputeDedupKey(bookmark.Text, "", position)
}
h.annotationSvc.SaveBookmark(ctx, wsync.SaveBookmarkRequest{
MediaItemID: mediaItemID,
UserID: userID,
@@ -559,8 +742,33 @@ func (h *KOReaderHandler) processBookAnnotations(ctx context.Context, deviceID,
ChapterNumber: int32(bookmark.Chapter),
Source: "koreader",
DeviceSyncData: deviceData,
DedupKey: dedupKey,
})
}
// Device-reported deletions: tombstone by dedup key. Tombstoned rows stay
// in the history (restorable from the book page) and are echoed to other
// devices as tombstones on their next pull. A device replay that pushes a
// stale copy of the annotation cannot resurrect the tombstone (its save
// carries no modification timestamp newer than the delete). Deletions run
// after the upserts purely so a key present in both lists resolves to
// "deleted" — the newer intent.
for _, del := range book.DeletedHighlights {
if del.DedupKey == "" {
continue
}
if err := h.annotationSvc.TombstoneHighlight(ctx, userID, mediaItemID, del.DedupKey, "koreader"); err != nil {
log.Printf("KOReader: tombstone highlight by dedup key failed: %v", err)
}
}
for _, del := range book.DeletedBookmarks {
if del.DedupKey == "" {
continue
}
if err := h.annotationSvc.TombstoneBookmarkByDedupKey(ctx, userID, mediaItemID, del.DedupKey, "koreader"); err != nil {
log.Printf("KOReader: tombstone bookmark by dedup key failed: %v", err)
}
}
}
func (h *KOReaderHandler) updateProgressForBook(c *echo.Context, deviceID pgtype.UUID, userID pgtype.UUID, mediaItemID pgtype.UUID, book KOReaderBookProgress) error {
@@ -710,6 +918,43 @@ func int64PtrToPgInt8(i *int64) pgtype.Int8 {
return pgtype.Int8{}
}
type KOReaderResolveResponse struct {
BookUUID string `json:"book_uuid"`
SHA256 string `json:"sha256,omitempty"`
Title string `json:"title,omitempty"`
Author string `json:"author,omitempty"`
}
// ResolveBook maps a file SHA-256 to the book's UUID without touching any
// progress state. Devices need the UUID to pull metadata, but a freshly
// downloaded book has none cached yet — the old way of learning it was to
// push once, which transmitted the device's first-page position to the
// server and manufactured a progress conflict for books already mid-read
// from another source. This read-only lookup lets the client link (and
// pull) without ever pushing bootstrap progress.
func (h *KOReaderHandler) ResolveBook(c *echo.Context) error {
sha256 := c.QueryParam("sha256")
if sha256 == "" || len(sha256) != 64 {
return c.JSON(http.StatusBadRequest, map[string]string{
"error": "sha256 query parameter is required (64 hex characters)",
})
}
mediaItem, _, err := h.bookResolver.ResolveBySHA256(c.Request().Context(), sha256)
if err != nil {
return c.JSON(http.StatusNotFound, map[string]string{
"error": "book not found",
})
}
return c.JSON(http.StatusOK, KOReaderResolveResponse{
BookUUID: uuid.UUID(mediaItem.ID.Bytes).String(),
SHA256: sha256,
Title: mediaItem.Title,
Author: mediaItem.Author.String,
})
}
func (h *KOReaderHandler) GetMetadata(c *echo.Context) error {
device := c.Get("device").(database.Devices)
userID := device.UserID.Bytes
@@ -798,34 +1043,52 @@ func (h *KOReaderHandler) GetMetadata(c *echo.Context) error {
for _, ann := range annotations {
if ann.AnnotationType == "highlight" {
pos0 := ann.StartPosition.String
pos1 := ann.EndPosition.String
if ann.EpubcfiStart.Valid && ann.EpubcfiStart.String != "" {
if converted := h.reverseConvertCFI(c, mediaItem, ann.EpubcfiStart.String); converted != "" {
pos0 = converted
}
// Selection text doubles as the converter's text-search context.
pos0 := h.koreaderPos0(c, mediaItem, ann.StartPosition.String, ann.EpubcfiStart.String, ann.SelectionText)
pos1 := h.koreaderPos0(c, mediaItem, ann.EndPosition.String, ann.EpubcfiEnd.String, ann.SelectionText)
if pos0 == "" {
// Nothing the device could place — serving a locator it can't
// resolve would create junk bookmarks that re-push as
// duplicates, so skip instead.
log.Printf("Bookhoard: GetMetadata skip highlight %s (no resolvable pos0)", ann.ID)
continue
}
if ann.EpubcfiEnd.Valid && ann.EpubcfiEnd.String != "" {
if converted := h.reverseConvertCFI(c, mediaItem, ann.EpubcfiEnd.String); converted != "" {
pos1 = converted
}
// Old web highlights carry no end anchor, and converted range
// CFIs resolve to their start — either way pos1 collapses onto
// pos0 and the device paints a zero-width highlight. Derive the
// end by advancing the start's character offset by the length
// of the selected text.
if pos1 == "" || pos1 == pos0 {
pos1 = extendXPointerByLength(pos0, ann.SelectionText)
}
highlight := KOReaderHighlight{
Text: ann.SelectionText,
Pos0: pos0,
Pos1: pos1,
Color: ann.Color.String,
Text: ann.SelectionText,
Pos0: pos0,
Pos1: pos1,
// Web colors flow to the device, mapped to KOReader's named
// palette. Round-trip safety: the device suppresses the color
// when echoing un-edited applied entries (a pink→purple
// palette mismatch must not rewrite the stored hex), and an
// actual device edit pushes its color, which wins.
Color: mapColorToKOReader(ann.Color.String),
Datetime: ann.CreatedAt.Time.Format(time.RFC3339),
DedupKey: ann.DedupKey.String,
}
if ann.NoteText.Valid && ann.NoteText.String != "" {
highlight.Notes = ann.NoteText.String
}
annotationsResponse.Highlights = append(annotationsResponse.Highlights, highlight)
} else if ann.AnnotationType == "note" {
pos0 := h.koreaderPos0(c, mediaItem, ann.StartPosition.String, ann.EpubcfiStart.String, "")
if pos0 == "" {
log.Printf("Bookhoard: GetMetadata skip note %s (no resolvable pos0)", ann.ID)
continue
}
annotationsResponse.Notes = append(annotationsResponse.Notes, KOReaderNote{
Text: ann.SelectionText,
Pos0: ann.StartPosition.String,
Pos0: pos0,
Datetime: ann.CreatedAt.Time.Format(time.RFC3339),
DedupKey: ann.DedupKey.String,
})
}
}
@@ -835,26 +1098,28 @@ func (h *KOReaderHandler) GetMetadata(c *echo.Context) error {
UserID: pgUserID,
})
for _, bm := range bookmarks {
pos0 := bm.Position.String
if pos0 == "" && bm.CfiPosition.Valid {
pos0 = bm.CfiPosition.String
pos0 := h.koreaderPos0(c, mediaItem, bm.Position.String, bm.CfiPosition.String, "")
if pos0 == "" {
log.Printf("Bookhoard: GetMetadata skip bookmark %s (no resolvable pos0)", bm.ID)
continue
}
koreaderBookmark := KOReaderBookmark{
Text: bm.Title,
Pos0: pos0,
Pos1: pos0,
Datetime: bm.CreatedAt.Time.Format(time.RFC3339),
DedupKey: bm.DedupKey.String,
}
if bm.Notes.Valid && bm.Notes.String != "" {
koreaderBookmark.Notes = bm.Notes.String
}
if bm.ChapterNumber.Valid {
koreaderBookmark.Chapter = int(bm.ChapterNumber.Int32)
koreaderBookmark.Chapter = FlexInt(bm.ChapterNumber.Int32)
}
annotationsResponse.Bookmarks = append(annotationsResponse.Bookmarks, koreaderBookmark)
}
cutoff := pgtype.Timestamptz{Time: time.Now().Add(-wsync.TombstoneTTL), Valid: true}
cutoff := pgtype.Timestamptz{Time: time.Now().Add(-h.annotationSvc.ActiveTombstoneTTL()), Valid: true}
tombstones, _ := h.db.GetTombstonedAnnotationsForBook(c.Request().Context(), database.GetTombstonedAnnotationsForBookParams{
MediaItemID: pgBookUUID,
UserID: pgUserID,
@@ -869,6 +1134,14 @@ func (h *KOReaderHandler) GetMetadata(c *echo.Context) error {
dd = map[string]interface{}{}
}
dd["dedup_key"] = ts.DedupKey.String
// KOReader deletes by matching pos0. Device-pushed annotations carry
// it in device_sync_data; web-created ones don't (their locator is
// converted at serve time), so resolve it from the stored columns.
if dd["pos0"] == nil || dd["pos0"] == "" {
if pos0 := h.koreaderPos0(c, mediaItem, ts.StartPosition.String, ts.EpubcfiStart.String, ""); pos0 != "" {
dd["pos0"] = pos0
}
}
if ts.AnnotationType == "highlight" {
annotationsResponse.DeletedHighlights = append(annotationsResponse.DeletedHighlights, dd)
} else if ts.AnnotationType == "bookmark" {
@@ -883,6 +1156,7 @@ func (h *KOReaderHandler) GetMetadata(c *echo.Context) error {
metadata := KOReaderMetadata{
UUID: bookUUID.String(),
SHA256: mediaItem.FileSha256.String,
Title: mediaItem.Title,
Authors: []string{mediaItem.Author.String},
Progress: progressData,
@@ -927,7 +1201,7 @@ func (h *KOReaderHandler) convertCFIToXPointer(c *echo.Context, mediaItem databa
}
}
func (h *KOReaderHandler) reverseConvertCFI(c *echo.Context, mediaItem database.MediaItems, epubcfi string) string {
func (h *KOReaderHandler) reverseConvertCFI(c *echo.Context, mediaItem database.MediaItems, epubcfi string, contextText string) string {
if h.libraryService == nil || epubcfi == "" {
return ""
}
@@ -935,13 +1209,138 @@ func (h *KOReaderHandler) reverseConvertCFI(c *echo.Context, mediaItem database.
if err != nil || epubPath == "" {
return ""
}
loc := wsync.ConvertFromCanonical(wsync.LocatorSourceKOReader, epubcfi, 0, "", mediaItem.FormatGroup, epubPath, "")
loc := wsync.ConvertFromCanonical(wsync.LocatorSourceKOReader, epubcfi, 0, contextText, mediaItem.FormatGroup, epubPath, "")
if loc.Position != "" && loc.Position != epubcfi {
return loc.Position
}
return ""
}
// pdfRectAnchor is the JSON locator the web reader stores in epubcfi_start
// for PDF text highlights (page-fraction rects; page index is 0-based).
type pdfRectAnchor struct {
V int `json:"v"`
Page int `json:"page"`
Rects [][]float64 `json:"rects"`
}
// koreaderPos0 resolves a device-native KOReader pos0 from an annotation's
// stored locators, whatever the source. Resolution order:
//
// extendXPointerByLength advances a CRE xpointer's trailing text-node
// character offset by the rune length of text, so a highlight with only a
// start anchor still gets a plausible (non-collapsed) end for drawing.
// Overshooting the node just clamps on the device.
func extendXPointerByLength(xp, text string) string {
if xp == "" || text == "" {
return xp
}
i := strings.LastIndex(xp, ".")
if i < 0 {
return xp
}
off, err := strconv.Atoi(xp[i+1:])
if err != nil {
return xp
}
return xp[:i+1] + strconv.Itoa(off+utf8.RuneCountInString(text))
}
// KOReader paints highlight colors from a fixed set of names
// (Blitbuffer.HIGHLIGHT_COLORS); the web reader uses hex swatches. Map at
// the boundary so each side always receives something it can render;
// unmappable values fall back to each side's default (yellow).
var koreaderColorFromName = map[string]string{
"yellow": "#ffd54f",
"orange": "#ffd54f",
"green": "#a5d6a7",
"olive": "#a5d6a7",
"cyan": "#90caf9",
"blue": "#90caf9",
"purple": "#ce93d8",
"red": "#f48fb1",
}
// mapColorFromKOReader normalizes a device color name to a web hex
// swatch (default yellow) when ingesting device pushes.
func mapColorFromKOReader(name string) string {
if hex, ok := koreaderColorFromName[strings.ToLower(strings.TrimSpace(name))]; ok {
return hex
}
return "#ffd54f"
}
var koreaderColorFromHex = map[string]string{
"#ffd54f": "yellow",
"#a5d6a7": "green",
"#90caf9": "blue",
"#ce93d8": "purple",
"#f48fb1": "purple",
}
// mapColorToKOReader normalizes a web hex swatch to the nearest KOReader
// color name (default yellow) when serving to devices. Pink maps to purple
// (the palette's closest); round-trip drift is prevented on the device by
// suppressing echo colors for un-edited applied entries.
func mapColorToKOReader(hex string) string {
if name, ok := koreaderColorFromHex[strings.ToLower(strings.TrimSpace(hex))]; ok {
return name
}
return "yellow"
}
// 1. A device-native CRE xpointer ("/body/...") in startPosition wins —
// round-trip identical for KOReader-pushed annotations (converting the
// stored CFI instead could drift and duplicate on the device).
// 2. The web reader's PDF JSON anchor → bare page number (KOReader paging
// documents use the page number as pos0).
// 3. A stored EPUB CFI (epubcfi_start, or startPosition without the
// reader's "cfi:" prefix) → converted to a CRE xpointer, with
// contextText (the selection text) enabling the text-search fallback.
// 4. A "page:N" or bare-numeric position → the bare number.
//
// Returns "" when nothing usable exists; callers skip such annotations so
// devices never receive locators they cannot place.
func (h *KOReaderHandler) koreaderPos0(c *echo.Context, mediaItem database.MediaItems, startPosition, epubcfi, contextText string) string {
if wsync.IsCREXPointer(startPosition) {
return startPosition
}
if strings.HasPrefix(epubcfi, "{") {
var anchor pdfRectAnchor
if json.Unmarshal([]byte(epubcfi), &anchor) == nil && anchor.Page >= 0 {
return strconv.Itoa(anchor.Page)
}
}
cfi := epubcfi
if cfi == "" && strings.HasPrefix(startPosition, "cfi:") {
cfi = strings.TrimPrefix(startPosition, "cfi:")
}
if cfi != "" && wsync.IsStandardEPUBCFI(cfi) {
if converted := h.reverseConvertCFI(c, mediaItem, cfi, contextText); converted != "" {
return converted
}
// Conversion failed; fall through so numeric positions still work.
if wsync.IsCREXPointer(cfi) {
return cfi
}
}
if p := strings.TrimPrefix(startPosition, "page:"); p != "" && parsePageInt(p) >= 0 {
return p
}
return ""
}
func parsePageInt(s string) int64 {
var n int64
for _, r := range s {
if r < '0' || r > '9' {
return -1
}
n = n*10 + int64(r-'0')
}
return n
}
func (h *KOReaderHandler) GetLibrary(c *echo.Context) error {
device := c.Get("device").(database.Devices)
userID := device.UserID.Bytes
@@ -989,6 +1388,7 @@ func (h *KOReaderHandler) GetLibrary(c *echo.Context) error {
libraryBooks = append(libraryBooks, KOReaderLibraryBook{
UUID: uuid.UUID(item.ID.Bytes).String(),
SHA256: item.FileSha256.String,
Title: item.Title,
Author: item.Author.String,
ContentType: "6",
@@ -1045,8 +1445,8 @@ func (h *KOReaderHandler) SyncBookmarks(c *echo.Context) error {
}
pgBookUUID = pgtype.UUID{Bytes: bookUUID, Valid: true}
} else if req.BookSHA256 != "" && len(req.BookSHA256) == 64 {
// Use SHA-256 to find book
mediaItem, err := h.db.GetMediaItemBySHA256(ctx, pgtype.Text{String: req.BookSHA256, Valid: true})
// Use SHA-256 to find book (format-aware: also checks media_item_formats)
mediaItem, _, err := h.bookResolver.ResolveBySHA256(ctx, req.BookSHA256)
if err != nil {
return c.JSON(http.StatusNotFound, map[string]string{
"error": "book not found by SHA-256",
@@ -1068,7 +1468,7 @@ func (h *KOReaderHandler) SyncBookmarks(c *echo.Context) error {
// If bookmark has its own SHA-256, use it for matching
if bookmark.BookSHA256 != "" && len(bookmark.BookSHA256) == 64 {
mediaItem, err := h.db.GetMediaItemBySHA256(ctx, pgtype.Text{String: bookmark.BookSHA256, Valid: true})
mediaItem, _, err := h.bookResolver.ResolveBySHA256(ctx, bookmark.BookSHA256)
if err == nil {
mediaItemID = mediaItem.ID
}
@@ -1119,7 +1519,7 @@ func (h *KOReaderHandler) SyncBookmarks(c *echo.Context) error {
// If note has its own SHA-256, use it for matching
if note.BookSHA256 != "" && len(note.BookSHA256) == 64 {
mediaItem, err := h.db.GetMediaItemBySHA256(ctx, pgtype.Text{String: note.BookSHA256, Valid: true})
mediaItem, _, err := h.bookResolver.ResolveBySHA256(ctx, note.BookSHA256)
if err == nil {
mediaItemID = mediaItem.ID
}
@@ -1169,7 +1569,7 @@ func (h *KOReaderHandler) SyncBookmarks(c *echo.Context) error {
// If highlight has its own SHA-256, use it for matching
if highlight.BookSHA256 != "" && len(highlight.BookSHA256) == 64 {
mediaItem, err := h.db.GetMediaItemBySHA256(ctx, pgtype.Text{String: highlight.BookSHA256, Valid: true})
mediaItem, _, err := h.bookResolver.ResolveBySHA256(ctx, highlight.BookSHA256)
if err == nil {
mediaItemID = mediaItem.ID
}
@@ -1182,13 +1582,13 @@ func (h *KOReaderHandler) SyncBookmarks(c *echo.Context) error {
endPos = startPos
}
color := "#ffff00"
color := "#ffd54f"
if highlight.Color != "" {
color = highlight.Color
color = mapColorFromKOReader(highlight.Color)
}
if h.annotationSvc != nil {
epubcfiStart, epubcfiEnd := h.convertHighlightPositions(ctx, mediaItemID, highlight.Pos0, highlight.Pos1)
epubcfiStart, epubcfiEnd := h.convertHighlightPositions(ctx, mediaItemID, highlight.Pos0, highlight.Pos1, highlight.Text)
pctStart := 0.0
if highlight.Percentage != nil {
+52
View File
@@ -0,0 +1,52 @@
package handlers
import (
"encoding/json"
"testing"
"github.com/stretchr/testify/assert"
)
// The device pushes deletions as dedup-key arrays on the progress request.
// Verify the wire shape the plugin sends (lua json.encode of
// { deleted_highlights = { { dedup_key = "..." } } }) binds correctly.
func TestKOReaderProgressRequest_DeletedAnnotationsBinding(t *testing.T) {
payload := `{
"books": [{
"sha256": "d1b1c6123d6206017b40798744ed994f00803b97d22ce51bea32e95e1ce7a164",
"title": "1984",
"percentage": 0.42,
"deleted_highlights": [
{ "dedup_key": "abc123" },
{ "dedup_key": "def456" }
],
"deleted_bookmarks": [
{ "dedup_key": "789xyz" }
]
}]
}`
var req KOReaderProgressRequest
err := json.Unmarshal([]byte(payload), &req)
assert.NoError(t, err)
assert.Len(t, req.Books, 1)
book := req.Books[0]
assert.Len(t, book.DeletedHighlights, 2)
assert.Equal(t, "abc123", book.DeletedHighlights[0].DedupKey)
assert.Equal(t, "def456", book.DeletedHighlights[1].DedupKey)
assert.Len(t, book.DeletedBookmarks, 1)
assert.Equal(t, "789xyz", book.DeletedBookmarks[0].DedupKey)
}
// A request without the arrays (older plugins) must bind with them empty —
// deletion propagation is strictly opt-in per push.
func TestKOReaderProgressRequest_DeletedAnnotationsOmitted(t *testing.T) {
payload := `{"books": [{"sha256": "x", "title": "t", "percentage": 0.1}]}`
var req KOReaderProgressRequest
err := json.Unmarshal([]byte(payload), &req)
assert.NoError(t, err)
assert.Empty(t, req.Books[0].DeletedHighlights)
assert.Empty(t, req.Books[0].DeletedBookmarks)
}
+241 -18
View File
@@ -115,20 +115,51 @@ type UpdateMediaNoteRequest struct {
// CreateMediaHighlightRequest represents the request for creating a media highlight
type CreateMediaHighlightRequest struct {
SelectionText string `json:"selection_text" validate:"required,min=1,max=5000"`
StartPosition string `json:"start_position" validate:"required,max=100"`
EndPosition string `json:"end_position" validate:"required,max=100"`
Color string `json:"color" validate:"omitempty,len=7"`
NoteID string `json:"note_id"`
SelectionText string `json:"selection_text" validate:"required,min=1,max=5000"`
StartPosition string `json:"start_position" validate:"max=1000"`
EndPosition string `json:"end_position" validate:"max=1000"`
EpubcfiStart string `json:"epubcfi_start" validate:"max=2000"`
EpubcfiEnd string `json:"epubcfi_end" validate:"max=2000"`
Color string `json:"color" validate:"omitempty,len=7"`
NoteText string `json:"note_text" validate:"max=10000"`
NoteID string `json:"note_id"`
PercentageStart float64 `json:"percentage_start"`
PercentageEnd float64 `json:"percentage_end"`
ChapterReference int32 `json:"chapter_reference"`
}
// UpdateMediaHighlightRequest represents the request for updating a media highlight
type UpdateMediaHighlightRequest struct {
SelectionText string `json:"selection_text" validate:"required,min=1,max=5000"`
StartPosition string `json:"start_position" validate:"required,max=100"`
EndPosition string `json:"end_position" validate:"required,max=100"`
Color string `json:"color" validate:"omitempty,len=7"`
NoteID string `json:"note_id"`
SelectionText string `json:"selection_text" validate:"required,min=1,max=5000"`
StartPosition string `json:"start_position" validate:"max=1000"`
EndPosition string `json:"end_position" validate:"max=1000"`
EpubcfiStart string `json:"epubcfi_start" validate:"max=2000"`
EpubcfiEnd string `json:"epubcfi_end" validate:"max=2000"`
Color string `json:"color" validate:"omitempty,len=7"`
NoteText string `json:"note_text" validate:"max=10000"`
NoteID string `json:"note_id"`
PercentageStart float64 `json:"percentage_start"`
PercentageEnd float64 `json:"percentage_end"`
ChapterReference int32 `json:"chapter_reference"`
}
// CreateMediaBookmarkRequest represents the request for creating a media bookmark
type CreateMediaBookmarkRequest struct {
Title string `json:"title" validate:"required,min=1,max=255"`
Position string `json:"position" validate:"max=100"`
Notes string `json:"notes" validate:"max=10000"`
CfiPosition string `json:"cfi_position" validate:"max=255"`
PageNumber int32 `json:"page_number"`
ChapterNumber int32 `json:"chapter_number"`
Percentage float64 `json:"percentage"`
ChapterReference int32 `json:"chapter_reference"`
}
// UpdateMediaBookmarkRequest represents the request for updating a media bookmark
type UpdateMediaBookmarkRequest struct {
Title string `json:"title" validate:"required,min=1,max=255"`
Notes string `json:"notes" validate:"max=10000"`
Position string `json:"position" validate:"max=100"`
}
type MediaHandler struct {
@@ -1557,14 +1588,20 @@ func (mh *MediaHandler) CreateMediaHighlight(c *echo.Context) error {
if mh.annotationSvc != nil {
result, err := mh.annotationSvc.SaveHighlight(c.Request().Context(), wsync.SaveHighlightRequest{
MediaItemID: pgMediaID,
UserID: pgUserID,
SelectionText: req.SelectionText,
StartPosition: req.StartPosition,
EndPosition: req.EndPosition,
Color: color,
Source: "web",
ModifiedAt: time.Now(),
MediaItemID: pgMediaID,
UserID: pgUserID,
SelectionText: req.SelectionText,
StartPosition: req.StartPosition,
EndPosition: req.EndPosition,
EpubcfiStart: req.EpubcfiStart,
EpubcfiEnd: req.EpubcfiEnd,
Color: color,
NoteText: req.NoteText,
PercentageStart: req.PercentageStart,
PercentageEnd: req.PercentageEnd,
ChapterReference: req.ChapterReference,
Source: "web",
ModifiedAt: time.Now(),
})
if err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
@@ -1637,6 +1674,42 @@ func (mh *MediaHandler) UpdateMediaHighlight(c *echo.Context) error {
color = req.Color
}
// Prefer the sync-aware path: the same selection text + CFI resolves to
// the same dedup key, so this performs an LWW update of the existing row
// (including note_text and CFI columns the plain query cannot touch).
if mh.annotationSvc != nil {
userID := c.Get("user_id").(string)
userUUID, err := uuid.Parse(userID)
if err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid user"})
}
mediaID := c.Param("id")
mediaUUID, err := uuid.Parse(mediaID)
if err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid media item id"})
}
result, err := mh.annotationSvc.SaveHighlight(c.Request().Context(), wsync.SaveHighlightRequest{
MediaItemID: pgtype.UUID{Bytes: mediaUUID, Valid: true},
UserID: pgtype.UUID{Bytes: userUUID, Valid: true},
SelectionText: req.SelectionText,
StartPosition: req.StartPosition,
EndPosition: req.EndPosition,
EpubcfiStart: req.EpubcfiStart,
EpubcfiEnd: req.EpubcfiEnd,
Color: color,
NoteText: req.NoteText,
PercentageStart: req.PercentageStart,
PercentageEnd: req.PercentageEnd,
ChapterReference: req.ChapterReference,
Source: "web",
ModifiedAt: time.Now(),
})
if err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
}
return c.JSON(http.StatusOK, result.Highlight)
}
highlight, err := mh.db.UpdateMediaHighlight(c.Request().Context(), database.UpdateMediaHighlightParams{
ID: pgtype.UUID{Bytes: highlightUUID, Valid: true},
SelectionText: req.SelectionText,
@@ -1677,6 +1750,152 @@ func (mh *MediaHandler) DeleteMediaHighlight(c *echo.Context) error {
return c.NoContent(http.StatusNoContent)
}
// GetMediaBookmarks handles GET /api/media-items/:id/bookmarks
func (mh *MediaHandler) GetMediaBookmarks(c *echo.Context) error {
userID := c.Get("user_id").(string)
userUUID, err := uuid.Parse(userID)
if err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid user"})
}
mediaID := c.Param("id")
mediaUUID, err := uuid.Parse(mediaID)
if err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid media item id"})
}
bookmarks, err := mh.db.GetMediaBookmarks(c.Request().Context(), database.GetMediaBookmarksParams{
MediaItemID: pgtype.UUID{Bytes: mediaUUID, Valid: true},
UserID: pgtype.UUID{Bytes: userUUID, Valid: true},
})
if err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
}
return c.JSON(http.StatusOK, bookmarks)
}
// CreateMediaBookmark handles POST /api/media-items/:id/bookmarks
func (mh *MediaHandler) CreateMediaBookmark(c *echo.Context) error {
userID := c.Get("user_id").(string)
userUUID, err := uuid.Parse(userID)
if err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid user"})
}
mediaID := c.Param("id")
mediaUUID, err := uuid.Parse(mediaID)
if err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid media item id"})
}
var req CreateMediaBookmarkRequest
if err := c.Bind(&req); err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid request"})
}
if err := c.Validate(&req); err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": err.Error()})
}
// The sync-aware path (dedup + LWW + tombstones) is preferred; fall back
// to the plain query when the service isn't wired (e.g. some tests).
if mh.annotationSvc != nil {
result, err := mh.annotationSvc.SaveBookmark(c.Request().Context(), wsync.SaveBookmarkRequest{
MediaItemID: pgtype.UUID{Bytes: mediaUUID, Valid: true},
UserID: pgtype.UUID{Bytes: userUUID, Valid: true},
Title: req.Title,
Position: req.Position,
Notes: req.Notes,
PageNumber: req.PageNumber,
ChapterNumber: req.ChapterNumber,
CFIPosition: req.CfiPosition,
PercentageLoc: req.Percentage,
ChapterReference: req.ChapterReference,
Source: "web",
ModifiedAt: time.Now(),
})
if err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
}
return c.JSON(http.StatusCreated, result.Bookmark)
}
bookmark, err := mh.db.CreateMediaBookmark(c.Request().Context(), database.CreateMediaBookmarkParams{
MediaItemID: pgtype.UUID{Bytes: mediaUUID, Valid: true},
UserID: pgtype.UUID{Bytes: userUUID, Valid: true},
PageNumber: pgtype.Int4{Int32: req.PageNumber, Valid: req.PageNumber > 0},
ChapterNumber: pgtype.Int4{Int32: req.ChapterNumber, Valid: req.ChapterNumber > 0},
CfiPosition: pgtype.Text{String: req.CfiPosition, Valid: req.CfiPosition != ""},
Title: req.Title,
Position: pgtype.Text{String: req.Position, Valid: req.Position != ""},
Notes: pgtype.Text{String: req.Notes, Valid: req.Notes != ""},
})
if err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
}
return c.JSON(http.StatusCreated, bookmark)
}
// UpdateMediaBookmark handles PUT /api/media-items/:id/bookmarks/:bookmarkId
func (mh *MediaHandler) UpdateMediaBookmark(c *echo.Context) error {
userID := c.Get("user_id").(string)
userUUID, err := uuid.Parse(userID)
if err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid user"})
}
bookmarkID := c.Param("bookmarkId")
bookmarkUUID, err := uuid.Parse(bookmarkID)
if err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid bookmark id"})
}
var req UpdateMediaBookmarkRequest
if err := c.Bind(&req); err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid request"})
}
if err := c.Validate(&req); err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": err.Error()})
}
bookmark, err := mh.db.UpdateMediaBookmark(c.Request().Context(), database.UpdateMediaBookmarkParams{
ID: pgtype.UUID{Bytes: bookmarkUUID, Valid: true},
Title: req.Title,
Notes: pgtype.Text{String: req.Notes, Valid: req.Notes != ""},
Position: pgtype.Text{String: req.Position, Valid: req.Position != ""},
UserID: pgtype.UUID{Bytes: userUUID, Valid: true},
})
if err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
}
return c.JSON(http.StatusOK, bookmark)
}
// DeleteMediaBookmark handles DELETE /api/media-items/:id/bookmarks/:bookmarkId
func (mh *MediaHandler) DeleteMediaBookmark(c *echo.Context) error {
bookmarkID := c.Param("bookmarkId")
bookmarkUUID, err := uuid.Parse(bookmarkID)
if err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid bookmark id"})
}
pgBookmarkID := pgtype.UUID{Bytes: bookmarkUUID, Valid: true}
if mh.annotationSvc != nil {
if err := mh.annotationSvc.TombstoneBookmarkByID(c.Request().Context(), pgBookmarkID, "web"); err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
}
return c.NoContent(http.StatusNoContent)
}
if err := mh.db.DeleteMediaBookmark(c.Request().Context(), pgBookmarkID); err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
}
return c.NoContent(http.StatusNoContent)
}
// SearchMediaItems handles GET /api/media-items/search
// Supports two modes:
// 1. Autocomplete: author=value, genre=value, etc. → returns field values for dropdowns
@@ -1791,6 +2010,10 @@ func (mh *MediaHandler) SearchMediaItems(c *echo.Context) error {
"results": []interface{}{},
})
}
for i := range results {
resolved := utils.ResolveMediaURL(results[i].LibraryID, results[i].CoverImagePath)
results[i].CoverImagePath = pgtype.Text{String: resolved, Valid: resolved != ""}
}
return c.JSON(http.StatusOK, results)
}
+4
View File
@@ -18,4 +18,8 @@ type MediaDetail struct {
// Computed counts
NotesCount int `json:"notes_count"`
HighlightsCount int `json:"highlights_count"`
// Deleted-annotation history (tombstoned rows, newest first) — the book
// page's "recently deleted" list with restore/permanent-delete actions.
DeletedAnnotations []DeletedAnnotationResponse `json:"deleted_annotations"`
}
+111 -24
View File
@@ -26,6 +26,26 @@ type OPDSHandler struct {
conversionService interface {
ConvertEPUBToKEPUB(ctx context.Context, mediaItemID pgtype.UUID, epubPath string) (*services.ConvertedKEPUB, error)
}
settings *database.SettingsRegistry
}
// SetSettings wires the tunable settings registry (OPDS page size).
func (h *OPDSHandler) SetSettings(s *database.SettingsRegistry) { h.settings = s }
// opdsDefaultPageSize returns the configured default page size (50 if unset).
func (h *OPDSHandler) opdsDefaultPageSize() int {
if h.settings != nil {
return h.settings.OpdsDefaultPageSize()
}
return 50
}
// opdsMaxPageSize returns the configured maximum page size (200 if unset).
func (h *OPDSHandler) opdsMaxPageSize() int {
if h.settings != nil {
return h.settings.OpdsMaxPageSize()
}
return 200
}
func NewOPDSHandler(db *database.Queries, libraryService *services.LibraryService, conversionService interface {
@@ -38,15 +58,15 @@ func NewOPDSHandler(db *database.Queries, libraryService *services.LibraryServic
}
}
// Helper function to get base URL from system config
// Helper function to get base URL from system config with request-derived fallback
func (h *OPDSHandler) getBaseURLs(c *echo.Context) (string, string, error) {
baseURL, err := h.db.GetSystemConfig(c.Request().Context(), "base_url")
if err != nil {
return "", "", fmt.Errorf("failed to get base_url from config: %w", err)
var dbBaseURL string
if config, err := h.db.GetSystemConfig(c.Request().Context(), "base_url"); err == nil {
dbBaseURL = config.Value
}
opdsBaseURL := baseURL.Value + "/opds"
return baseURL.Value, opdsBaseURL, nil
baseURL := deriveBaseURL(c, dbBaseURL)
opdsBaseURL := baseURL + "/opds"
return baseURL, opdsBaseURL, nil
}
func (h *OPDSHandler) getAuthToken(c *echo.Context) string {
@@ -67,6 +87,42 @@ func appendToken(url, token string) string {
return url + "?token=" + token
}
// catalogMediaType is the OPDS media type for an acquisition catalog feed.
const catalogMediaType = "application/atom+xml;profile=opds-catalog;kind=acquisition"
// addCatalogPaginationLinks adds OPDS pagination links (self, start, first,
// previous, next, last) and OpenSearch paging metadata (totalResults,
// itemsPerPage, startIndex) to a feed based on the current page position.
// catalogBase is the device catalog URL without query parameters. The token
// (device auth) is appended to every generated link.
func addCatalogPaginationLinks(feed *opds.Feed, catalogBase string, pageNum, perPageNum, totalItems int, token string) {
totalPages := 0
if totalItems > 0 {
totalPages = (totalItems + perPageNum - 1) / perPageNum
}
startIdx := (pageNum - 1) * perPageNum
pagedURL := func(page int) string {
return appendToken(fmt.Sprintf("%s?page=%d&per_page=%d", catalogBase, page, perPageNum), token)
}
// self reflects the current page; start/first point to the first page
feed.AddLink(pagedURL(pageNum), catalogMediaType, "self")
feed.AddLink(pagedURL(1), catalogMediaType, "start")
feed.AddLink(pagedURL(1), catalogMediaType, "first")
if totalPages > 0 {
feed.AddLink(pagedURL(totalPages), catalogMediaType, "last")
}
if pageNum > 1 {
feed.AddLink(pagedURL(pageNum-1), catalogMediaType, "previous")
}
if pageNum < totalPages {
feed.AddLink(pagedURL(pageNum+1), catalogMediaType, "next")
}
feed.SetPagination(totalItems, perPageNum, startIdx+1)
}
// resolveMimeType returns the mime type for a media item, preferring the stored
// mime_type, then format_mimetype, and finally falling back to EPUB.
func resolveMimeType(mime, formatMime pgtype.Text) string {
@@ -132,9 +188,10 @@ func (h *OPDSHandler) GetDeviceCatalog(c *echo.Context) error {
}
}
perPageNum := 50
perPageNum := h.opdsDefaultPageSize()
maxPerPage := h.opdsMaxPageSize()
if perPage != "" {
if num, err := strconv.Atoi(perPage); err == nil && num > 0 && num <= 200 {
if num, err := strconv.Atoi(perPage); err == nil && num > 0 && num <= maxPerPage {
perPageNum = num
}
}
@@ -206,14 +263,17 @@ func (h *OPDSHandler) GetDeviceCatalog(c *echo.Context) error {
"Bookhoard Library",
)
// Add feed links
// Feed links, including OPDS pagination links (first/previous/next/last) and
// OpenSearch paging metadata (totalResults/itemsPerPage/startIndex).
token := h.getAuthToken(c)
catalogURL := appendToken(fmt.Sprintf("%s/devices/%s/catalog", opdsBaseURL, deviceID), token)
feed.AddLink(catalogURL, "application/atom+xml;profile=opds-catalog;kind=acquisition", "self")
feed.AddLink(catalogURL, "application/atom+xml;profile=opds-catalog;kind=acquisition", "start")
catalogBase := fmt.Sprintf("%s/devices/%s/catalog", opdsBaseURL, deviceID)
addCatalogPaginationLinks(feed, catalogBase, pageNum, perPageNum, totalItems, token)
// OpenSearch: the search link points to an OpenSearch description document
// (served by the same /search endpoint when no query is supplied) so that
// OPDS clients like KOReader can discover how to formulate search requests.
searchURL := appendToken(fmt.Sprintf("%s/devices/%s/search", opdsBaseURL, deviceID), token)
feed.AddLink(searchURL, "application/atom+xml;profile=opds-catalog;kind=acquisition", "search")
feed.AddLink(searchURL, "application/opensearchdescription+xml", "search")
// Add entries
for _, item := range allItems {
@@ -286,16 +346,17 @@ func (h *OPDSHandler) GetDeviceCatalog(c *echo.Context) error {
return c.String(http.StatusOK, xmlString)
}
// SearchDeviceCatalog searches the OPDS catalog for a device
// SearchDeviceCatalog searches the OPDS catalog for a device.
//
// When no "q" query parameter is supplied it returns an OpenSearch description
// document (application/opensearchdescription+xml) so that OPDS clients such as
// KOReader can discover the search URL template (which contains the
// {searchTerms} placeholder). When "q" is supplied it returns an OPDS
// acquisition feed of matching books.
func (h *OPDSHandler) SearchDeviceCatalog(c *echo.Context) error {
deviceID := c.Param("deviceId")
query := c.QueryParam("q")
if query == "" {
return c.XML(http.StatusBadRequest, opds.NewErrorFeed("Missing search query"))
}
// Get base URLs
baseURL, opdsBaseURL, err := h.getBaseURLs(c)
if err != nil {
@@ -316,12 +377,30 @@ func (h *OPDSHandler) SearchDeviceCatalog(c *echo.Context) error {
// Get user's visible libraries
userID := device.UserID.Bytes
_, err = h.db.GetUserVisibleLibraries(c.Request().Context(), pgtype.UUID{Bytes: userID, Valid: true})
if err != nil {
return c.XML(http.StatusInternalServerError, opds.NewErrorFeed("Failed to get libraries"))
}
token := h.getAuthToken(c)
// No query: serve the OpenSearch description document so clients can learn
// the search template (contains the {searchTerms} placeholder).
if query == "" {
searchURL := appendToken(fmt.Sprintf("%s/devices/%s/search?q={searchTerms}", opdsBaseURL, deviceID), token)
desc := opds.NewSearchDescription(
"Bookhoard",
"Search the Bookhoard library",
searchURL,
)
xmlString, err := desc.GenerateXMLString()
if err != nil {
return c.XML(http.StatusInternalServerError, opds.NewErrorFeed("Failed to generate search description"))
}
c.Response().Header().Set("Content-Type", "application/opensearchdescription+xml")
return c.String(http.StatusOK, xmlString)
}
// Search media items
allItems, err := h.db.SearchMediaItems(c.Request().Context(), database.SearchMediaItemsParams{
UserID: pgtype.UUID{Bytes: userID, Valid: true},
@@ -340,12 +419,14 @@ func (h *OPDSHandler) SearchDeviceCatalog(c *echo.Context) error {
)
// Add feed links
token := h.getAuthToken(c)
catalogURL := appendToken(fmt.Sprintf("%s/devices/%s/catalog", opdsBaseURL, deviceID), token)
feed.AddLink(catalogURL, "application/atom+xml;profile=opds-catalog;kind=acquisition", "start")
feed.AddLink(catalogURL, catalogMediaType, "start")
searchURL := appendToken(fmt.Sprintf("%s/devices/%s/search?q=%s", opdsBaseURL, deviceID, query), token)
feed.AddLink(searchURL, "application/atom+xml;profile=opds-catalog;kind=acquisition", "self")
feed.AddLink(searchURL, catalogMediaType, "self")
// OpenSearch paging metadata (search results are a single page)
feed.SetPagination(len(allItems), len(allItems), 1)
// Add entries (same as catalog)
userUUID := uuid.UUID(userID)
@@ -524,6 +605,12 @@ func (h *OPDSHandler) DownloadBook(c *echo.Context) error {
if mediaItem.MimeType.Valid {
mimeType = mediaItem.MimeType.String
}
// Always expose the primary content hash so clients (e.g. the koreader
// plugin) learn the canonical SHA-256 from the download response itself,
// not just from the feed metadata.
if mediaItem.FileSha256.Valid {
fileSha256 = mediaItem.FileSha256.String
}
}
// Check if file exists
+119
View File
@@ -0,0 +1,119 @@
package handlers
import (
"bookhoard/internal/opds"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// rels collects the rel attributes of all links currently on the feed.
func rels(feed *opds.Feed) []string {
out := make([]string, 0, len(feed.Links))
for _, l := range feed.Links {
out = append(out, l.Rel)
}
return out
}
func containsRel(feed *opds.Feed, rel string) bool {
for _, l := range feed.Links {
if l.Rel == rel {
return true
}
}
return false
}
func TestAddCatalogPaginationLinks_MiddlePage(t *testing.T) {
feed := opds.NewFeed("urn:uuid:dev", "Library")
// 1814 items, 50 per page => 37 pages; on page 2
addCatalogPaginationLinks(feed, "http://h/opds/devices/dev/catalog", 2, 50, 1814, "tok")
assert.True(t, containsRel(feed, "self"))
assert.True(t, containsRel(feed, "start"))
assert.True(t, containsRel(feed, "first"))
assert.True(t, containsRel(feed, "last"))
assert.True(t, containsRel(feed, "previous"), "middle page must have previous")
assert.True(t, containsRel(feed, "next"), "middle page must have next")
// self must point to the current page
var selfHref string
for _, l := range feed.Links {
if l.Rel == "self" {
selfHref = l.Href
}
}
assert.Contains(t, selfHref, "page=2&per_page=50")
assert.Contains(t, selfHref, "token=tok")
// next must advance the page
var nextHref string
for _, l := range feed.Links {
if l.Rel == "next" {
nextHref = l.Href
}
}
assert.Contains(t, nextHref, "page=3")
// OpenSearch metadata
require.NotNil(t, feed.TotalResults)
assert.Equal(t, 1814, *feed.TotalResults)
require.NotNil(t, feed.ItemsPerPage)
assert.Equal(t, 50, *feed.ItemsPerPage)
require.NotNil(t, feed.StartIndex)
assert.Equal(t, 51, *feed.StartIndex, "startIndex should be 1-based offset of first item on page 2")
}
func TestAddCatalogPaginationLinks_FirstPage_NoPrevious(t *testing.T) {
feed := opds.NewFeed("urn:uuid:dev", "Library")
addCatalogPaginationLinks(feed, "http://h/opds/devices/dev/catalog", 1, 50, 1814, "")
rels := rels(feed)
assert.NotContains(t, rels, "previous", "first page must not have previous")
assert.Contains(t, rels, "next")
}
func TestAddCatalogPaginationLinks_LastPage_NoNext(t *testing.T) {
feed := opds.NewFeed("urn:uuid:dev", "Library")
addCatalogPaginationLinks(feed, "http://h/opds/devices/dev/catalog", 37, 50, 1814, "")
rels := rels(feed)
assert.NotContains(t, rels, "next", "last page must not have next")
assert.Contains(t, rels, "previous")
}
func TestAddCatalogPaginationLinks_SinglePage(t *testing.T) {
feed := opds.NewFeed("urn:uuid:dev", "Library")
addCatalogPaginationLinks(feed, "http://h/opds/devices/dev/catalog", 1, 50, 10, "")
rels := rels(feed)
assert.NotContains(t, rels, "previous")
assert.NotContains(t, rels, "next")
// still emits self/start/first/last
assert.Contains(t, rels, "self")
assert.Contains(t, rels, "last")
}
func TestAddCatalogPaginationLinks_EmptyCatalog(t *testing.T) {
feed := opds.NewFeed("urn:uuid:dev", "Library")
addCatalogPaginationLinks(feed, "http://h/opds/devices/dev/catalog", 1, 50, 0, "")
rels := rels(feed)
assert.NotContains(t, rels, "next")
assert.NotContains(t, rels, "previous")
assert.NotContains(t, rels, "last", "empty catalog should not advertise a last page")
require.NotNil(t, feed.TotalResults)
assert.Equal(t, 0, *feed.TotalResults)
}
func TestAddCatalogPaginationLinks_TokenAppended(t *testing.T) {
feed := opds.NewFeed("urn:uuid:dev", "Library")
addCatalogPaginationLinks(feed, "http://h/opds/devices/dev/catalog", 1, 50, 100, "abc")
xml, err := feed.GenerateXMLString()
require.NoError(t, err)
assert.True(t, strings.Count(xml, "token=abc") >= 3, "token should be appended to generated links")
}
+6
View File
@@ -2,6 +2,7 @@ package handlers
import (
"bookhoard/internal/database"
"context"
"net/http"
"time"
@@ -139,3 +140,8 @@ func (h *ProcessingIssuesHandler) DeleteProcessingIssue(c *echo.Context) error {
"message": "Issue deleted",
})
}
// GetProcessingIssueStatsData returns stats for SSR (not JSON response)
func (h *ProcessingIssuesHandler) GetProcessingIssueStatsData(ctx context.Context, libraryID pgtype.UUID) (database.GetProcessingIssueStatsRow, error) {
return h.db.GetProcessingIssueStats(ctx, libraryID)
}
+10
View File
@@ -374,6 +374,11 @@ func (h *Handler) GetAllProgressData(c *echo.Context) ([]ProgressWithMedia, erro
deviceName = progress.LastSyncDevice.String
}
lastUpdated := ""
if progress.LastReadAt.Valid {
lastUpdated = progress.LastReadAt.Time.Format("01-02-2006 03:04 PM")
}
progressList = append(progressList, ProgressWithMedia{
MediaItemID: progress.MediaItemID.Bytes,
Title: mediaItem.Title,
@@ -386,6 +391,11 @@ func (h *Handler) GetAllProgressData(c *echo.Context) ([]ProgressWithMedia, erro
Epubcfi: epubcfi,
LastSyncDevice: deviceName,
ProgressPercentage: progress.Percentage.Float64 * 100,
EpubCFI: epubcfi,
LastUpdated: lastUpdated,
DeviceIcon: getDeviceIcon(deviceName),
DeviceName: deviceName,
DeviceType: deviceName,
FormatGroup: mediaItem.FormatGroup,
EstimatedPages: wsync.EstimatedPages(mediaItem.TotalCharacters.Int64),
})
+2 -6
View File
@@ -14,10 +14,6 @@ import (
"github.com/labstack/echo/v5"
)
const (
refreshTokenExpiration = 7 * 24 * time.Hour // 7 days
)
type RefreshTokenRequest struct {
RefreshToken string `json:"refresh_token" validate:"required"`
}
@@ -72,7 +68,7 @@ func (h *AuthHandler) RefreshAccessToken(c *echo.Context) error {
return c.JSON(http.StatusOK, RefreshTokenResponse{
AccessToken: accessToken,
TokenType: "Bearer",
ExpiresIn: SessionDurationSec,
ExpiresIn: int(h.refreshTokenTTL().Seconds()),
})
}
@@ -101,7 +97,7 @@ func (h *AuthHandler) CreateRefreshToken(userID uuid.UUID) (string, string, erro
tokenUUID := uuid.New()
refreshToken := tokenUUID.String()
expiresAt := time.Now().Add(refreshTokenExpiration)
expiresAt := time.Now().Add(h.refreshTokenTTL())
_, err := h.db.CreateRefreshToken(context.Background(), database.CreateRefreshTokenParams{
UserID: pgtype.UUID{Bytes: userID, Valid: true},
Token: pgtype.UUID{Bytes: tokenUUID, Valid: true},
+3 -3
View File
@@ -128,8 +128,8 @@ func (h *Handler) StartScanner(c *echo.Context) error {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid user id"})
}
// Set the folder paths
if err := h.scanner.SetFolders(req.FolderPaths); err != nil {
// Set the folder paths (watch=true: this long-lived scanner reads events)
if err := h.scanner.SetFolders(req.FolderPaths, true); err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid folder paths: " + err.Error()})
}
@@ -202,7 +202,7 @@ func (h *Handler) StartWatchModeForLibrary(ctx context.Context, libraryID pgtype
}
scanner := services.NewMediaScanner(h.db)
if err := scanner.SetFolders(folderPaths); err != nil {
if err := scanner.SetFolders(folderPaths, true); err != nil {
return fmt.Errorf("failed to set scanner folders: %v", err)
}
+59 -20
View File
@@ -3,6 +3,7 @@ package handlers
import (
"bookhoard/internal/config"
"bookhoard/internal/database"
"bookhoard/internal/setupstatus"
"encoding/json"
"fmt"
"net/http"
@@ -14,14 +15,19 @@ import (
)
type SidecarHandler struct {
db *database.Queries
cfg *config.Config
db *database.Queries
cfg *config.Config
settings *database.SettingsRegistry
}
func NewSidecarHandler(db *database.Queries, cfg *config.Config) *SidecarHandler {
return &SidecarHandler{db: db, cfg: cfg}
}
// SetSettings wires the tunable settings registry so the timezone write path
// keeps the cache consistent.
func (h *SidecarHandler) SetSettings(s *database.SettingsRegistry) { h.settings = s }
type SidecarConfig struct {
Version string `json:"version"`
Bookhoard SidecarBookhoardConfig `json:"bookhoard"`
@@ -81,15 +87,13 @@ func (h *SidecarHandler) GetSidecarConfig(c *echo.Context) error {
userID := device.UserID.Bytes
pgUserID := pgtype.UUID{Bytes: userID, Valid: true}
// Get base URL and compute paths
baseURL, _ := h.db.GetSystemConfig(ctx, "base_url")
if baseURL.Value == "" {
baseURL.Value = h.cfg.BaseURL
}
// Get base URL and compute paths (with request-derived fallback)
dbBaseURL, _ := h.db.GetSystemConfig(ctx, "base_url")
baseURL := deriveBaseURL(c, dbBaseURL.Value)
// Generate URLs
opdsCatalogURL := fmt.Sprintf("%s/opds/devices/%s/catalog", baseURL.Value, deviceID.String())
syncAPIURL := fmt.Sprintf("%s/api/sync/kobo", baseURL.Value)
opdsCatalogURL := fmt.Sprintf("%s/opds/devices/%s/catalog", baseURL, deviceID.String())
syncAPIURL := fmt.Sprintf("%s/api/sync/kobo", baseURL)
// Get user's visible libraries with media items
mediaItems, err := h.db.GetUserMediaItemsForSync(ctx, pgUserID)
@@ -130,6 +134,21 @@ func (h *SidecarHandler) GetSidecarConfig(c *echo.Context) error {
SHA256: item.FileSha256.String,
FilePath: item.FilePath,
}
// Also key the book by each per-format hash (KEPUB/PDF/...), so a device
// holding a converted format resolves via the sidecar the same way it
// would via BookResolver on the server.
formats, ferr := h.db.GetMediaItemFormats(ctx, item.ID)
if ferr == nil {
entry := books[key]
for _, f := range formats {
if f.FileSha256.Valid && f.FileSha256.String != "" {
if _, exists := books[f.FileSha256.String]; !exists {
books[f.FileSha256.String] = entry
}
}
}
}
}
// Get collections
@@ -176,8 +195,8 @@ func (h *SidecarHandler) GetSidecarConfig(c *echo.Context) error {
Bookhoard: SidecarBookhoardConfig{
OPDSCatalog: opdsCatalogURL,
SyncAPI: syncAPIURL,
OPDSBaseURL: baseURL.Value + "/opds",
APIBaseURL: baseURL.Value + "/api",
OPDSBaseURL: baseURL + "/opds",
APIBaseURL: baseURL + "/api",
DeviceID: deviceID.String(),
DeviceToken: device.AuthToken,
},
@@ -219,15 +238,13 @@ func (h *SidecarHandler) DownloadSidecarConfig(c *echo.Context) error {
userID := device.UserID.Bytes
pgUserID := pgtype.UUID{Bytes: userID, Valid: true}
// Get base URL and compute paths
baseURL, _ := h.db.GetSystemConfig(ctx, "base_url")
if baseURL.Value == "" {
baseURL.Value = h.cfg.BaseURL
}
// Get base URL and compute paths (with request-derived fallback)
dbBaseURL, _ := h.db.GetSystemConfig(ctx, "base_url")
baseURL := deriveBaseURL(c, dbBaseURL.Value)
// Generate URLs
opdsCatalogURL := fmt.Sprintf("%s/opds/devices/%s/catalog", baseURL.Value, deviceID.String())
syncAPIURL := fmt.Sprintf("%s/api/sync/kobo", baseURL.Value)
opdsCatalogURL := fmt.Sprintf("%s/opds/devices/%s/catalog", baseURL, deviceID.String())
syncAPIURL := fmt.Sprintf("%s/api/sync/kobo", baseURL)
// Get user's visible libraries with media items
mediaItems, err := h.db.GetUserMediaItemsForSync(ctx, pgUserID)
@@ -267,6 +284,21 @@ func (h *SidecarHandler) DownloadSidecarConfig(c *echo.Context) error {
SHA256: item.FileSha256.String,
FilePath: item.FilePath,
}
// Also key the book by each per-format hash (KEPUB/PDF/...), so a device
// holding a converted format resolves via the sidecar the same way it
// would via BookResolver on the server.
formats, ferr := h.db.GetMediaItemFormats(ctx, item.ID)
if ferr == nil {
entry := books[key]
for _, f := range formats {
if f.FileSha256.Valid && f.FileSha256.String != "" {
if _, exists := books[f.FileSha256.String]; !exists {
books[f.FileSha256.String] = entry
}
}
}
}
}
// Get collections
@@ -309,8 +341,8 @@ func (h *SidecarHandler) DownloadSidecarConfig(c *echo.Context) error {
Bookhoard: SidecarBookhoardConfig{
OPDSCatalog: opdsCatalogURL,
SyncAPI: syncAPIURL,
OPDSBaseURL: baseURL.Value + "/opds",
APIBaseURL: baseURL.Value + "/api",
OPDSBaseURL: baseURL + "/opds",
APIBaseURL: baseURL + "/api",
DeviceID: deviceID.String(),
DeviceToken: device.AuthToken,
},
@@ -403,6 +435,9 @@ func (h *SidecarHandler) UpdateSystemConfiguration(c *echo.Context) error {
"error": "failed to update default timezone",
})
}
if h.settings != nil {
h.settings.Reload(ctx)
}
continue
}
_, err := h.db.SetSystemConfig(ctx, database.SetSystemConfigParams{
@@ -434,6 +469,10 @@ func (h *SidecarHandler) UpdateSystemConfiguration(c *echo.Context) error {
})
}
}
// Invalidate setup status cache so the middleware picks up the new
// base_url immediately (setup is not complete until base_url is set).
setupstatus.Invalidate()
}
// Check for HTMX request
+165 -1
View File
@@ -2,17 +2,21 @@ package handlers
import (
"bookhoard/internal/database"
"context"
"errors"
"fmt"
"net/http"
"strconv"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgtype"
"github.com/labstack/echo/v5"
)
type SystemSettingsHandler struct {
db *database.Queries
db *database.Queries
settings *database.SettingsRegistry
}
func NewSystemSettingsHandler(db *database.Queries) *SystemSettingsHandler {
@@ -21,6 +25,154 @@ func NewSystemSettingsHandler(db *database.Queries) *SystemSettingsHandler {
}
}
// SetSettings wires the tunable settings registry. Required for the unified
// /api/system/settings endpoints and for cache invalidation after writes.
func (h *SystemSettingsHandler) SetSettings(s *database.SettingsRegistry) {
h.settings = s
}
// reload refreshes the in-memory cache after a write.
func (h *SystemSettingsHandler) reload(c *echo.Context) {
if h.settings != nil {
h.settings.Reload(c.Request().Context())
}
}
// ---- Unified /api/system/settings endpoints ----
// GetSettings handles GET /api/system/settings.
func (h *SystemSettingsHandler) GetSettings(c *echo.Context) error {
if h.settings == nil {
return c.JSON(http.StatusServiceUnavailable, map[string]string{"error": "settings registry not initialized"})
}
return c.JSON(http.StatusOK, h.settings.All())
}
// UpdateSettingRequest is the body for PUT /api/system/settings.
type UpdateSettingRequest struct {
Key string `json:"key" form:"key"`
Value string `json:"value" form:"value"`
}
// UpdateSettingResponse mirrors a settings entry plus a reload hint.
type UpdateSettingResponse struct {
database.SettingEntry
ReloadRequired bool `json:"reload_required"`
Message string `json:"message,omitempty"`
}
// UpdateSetting handles PUT /api/system/settings.
func (h *SystemSettingsHandler) UpdateSetting(c *echo.Context) error {
if h.settings == nil {
return c.JSON(http.StatusServiceUnavailable, map[string]string{"error": "settings registry not initialized"})
}
var req UpdateSettingRequest
if err := c.Bind(&req); err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid request"})
}
resp, err := h.ApplySetting(c.Request().Context(), req.Key, req.Value)
if err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": err.Error()})
}
return c.JSON(http.StatusOK, resp)
}
// ApplySetting validates, persists, and reloads a single setting. Shared by the
// JSON API and the HTMX admin endpoint.
func (h *SystemSettingsHandler) ApplySetting(ctx context.Context, key, value string) (UpdateSettingResponse, error) {
if h.settings == nil {
return UpdateSettingResponse{}, fmt.Errorf("settings registry not initialized")
}
if key == "" {
return UpdateSettingResponse{}, fmt.Errorf("key is required")
}
def, ok := database.LookupDefault(key)
if !ok {
return UpdateSettingResponse{}, fmt.Errorf("unknown setting key: %s", key)
}
if err := validateSettingValue(def, value); err != nil {
return UpdateSettingResponse{}, err
}
desc := def.Description
rType := pgtype.Text{}
if def.Type != "" {
rType = pgtype.Text{String: def.Type, Valid: true}
}
var minP, maxP pgtype.Text
if def.Min != "" {
minP = pgtype.Text{String: def.Min, Valid: true}
}
if def.Max != "" {
maxP = pgtype.Text{String: def.Max, Valid: true}
}
if _, err := h.db.UpsertSystemSetting(ctx, database.UpsertSystemSettingParams{
SettingKey: key,
SettingValue: value,
Description: pgtype.Text{String: desc, Valid: desc != ""},
SettingType: rType,
MinValue: minP,
MaxValue: maxP,
RequiresRestart: pgtype.Bool{Bool: def.RequiresRestart, Valid: true},
Category: pgtype.Text{String: def.Category, Valid: def.Category != ""},
}); err != nil {
return UpdateSettingResponse{}, err
}
h.settings.Reload(ctx)
resp := UpdateSettingResponse{ReloadRequired: def.RequiresRestart}
for _, e := range h.settings.All() {
if e.Key == key {
resp.SettingEntry = e
break
}
}
if def.RequiresRestart {
resp.Message = "Saved. Restart the server for this change to take full effect."
} else {
resp.Message = "Saved."
}
return resp, nil
}
// validateSettingValue checks a candidate value against the setting's type and bounds.
func validateSettingValue(def database.SettingDefault, value string) error {
switch def.Type {
case database.SettingTypeInt:
n, err := strconv.Atoi(value)
if err != nil {
return fmt.Errorf("value must be an integer")
}
if def.Min != "" {
if mn, err := strconv.Atoi(def.Min); err == nil && n < mn {
return fmt.Errorf("value must be >= %s", def.Min)
}
}
if def.Max != "" {
if mx, err := strconv.Atoi(def.Max); err == nil && n > mx {
return fmt.Errorf("value must be <= %s", def.Max)
}
}
case database.SettingTypeBool:
if _, err := strconv.ParseBool(value); err != nil {
return fmt.Errorf("value must be true or false")
}
case database.SettingTypeString:
if value == "" {
return fmt.Errorf("value must not be empty")
}
if def.Key == "default_timezone" {
if _, err := time.LoadLocation(value); err != nil {
return fmt.Errorf("invalid timezone: %v", err)
}
}
}
return nil
}
// ---- Legacy scan-settings endpoints (retained for backward compatibility) ----
type UpdateScanSettingsRequest struct {
ScanPollIntervalSeconds int32 `json:"scan_poll_interval_seconds" validate:"required,min=1,max=3600"`
AutoScanEnabled bool `json:"auto_scan_enabled"`
@@ -51,6 +203,7 @@ func (h *SystemSettingsHandler) UpdateTimezoneSettings(c *echo.Context) error {
if err != nil {
return err
}
h.reload(c)
return c.JSON(http.StatusOK, map[string]string{"message": "Timezone updated"})
}
@@ -88,6 +241,8 @@ func (h *SystemSettingsHandler) UpdateScanSettings(c *echo.Context) error {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
}
h.reload(c)
return c.JSON(http.StatusOK, ScanSettingsResponse{
ScanPollIntervalSeconds: req.ScanPollIntervalSeconds,
AutoScanEnabled: req.AutoScanEnabled,
@@ -96,6 +251,15 @@ func (h *SystemSettingsHandler) UpdateScanSettings(c *echo.Context) error {
}
func (h *SystemSettingsHandler) GetScanSettings(c *echo.Context) error {
// Prefer the registry (single source of truth after Load).
if h.settings != nil {
interval := int32(h.settings.ScanPollInterval().Seconds())
return c.JSON(http.StatusOK, ScanSettingsResponse{
ScanPollIntervalSeconds: interval,
AutoScanEnabled: h.settings.AutoScanEnabled(),
})
}
scanFrequencySetting, err := h.db.GetSystemSetting(c.Request().Context(), "scan_poll_interval_seconds")
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
+36
View File
@@ -0,0 +1,36 @@
package handlers
import (
"strings"
"github.com/labstack/echo/v5"
)
// deriveBaseURL returns the base URL to use for constructing self-referential
// links (OPDS feeds, sidecar config, etc.). It prefers the database-configured
// base_url when available, and falls back to deriving the URL from the incoming
// HTTP request (Host header + scheme), which is always reachable by the client.
//
// Proxy header support: X-Forwarded-Proto and X-Forwarded-Host are respected so
// that deployments behind TLS-terminating reverse proxies advertise the correct
// external URL.
func deriveBaseURL(c *echo.Context, dbBaseURL string) string {
if dbBaseURL != "" {
return strings.TrimRight(dbBaseURL, "/")
}
scheme := "http"
if c.Request().TLS != nil {
scheme = "https"
}
if proto := c.Request().Header.Get("X-Forwarded-Proto"); proto != "" {
scheme = proto
}
host := c.Request().Host
if forwarded := c.Request().Header.Get("X-Forwarded-Host"); forwarded != "" {
host = forwarded
}
return scheme + "://" + host
}
+40 -7
View File
@@ -25,6 +25,7 @@ type DeviceContext struct {
type DeviceAuthMiddleware struct {
db *database.Queries
rateLimiter *DeviceRateLimiter
settings *database.SettingsRegistry
}
func NewDeviceAuthMiddleware(db *database.Queries) *DeviceAuthMiddleware {
@@ -34,6 +35,41 @@ func NewDeviceAuthMiddleware(db *database.Queries) *DeviceAuthMiddleware {
}
}
// SetSettings wires the tunable settings registry so device rate limits are
// read live on each authenticated request.
func (m *DeviceAuthMiddleware) SetSettings(s *database.SettingsRegistry) { m.settings = s }
// rateLimitConfig returns the active device rate limits from the registry, or
// the historical defaults when no registry is wired.
func (m *DeviceAuthMiddleware) rateLimitConfig() DeviceRateLimitConfig {
if m.settings != nil {
dl := m.settings.DeviceRateLimits()
return DeviceRateLimitConfig{
SyncRequestsPerMinute: dl.Sync,
ProgressUpdatesPerMinute: dl.Progress,
MetadataRequestsPerMinute: dl.Metadata,
}
}
return DeviceRateLimitConfig{
SyncRequestsPerMinute: DefaultSyncRequestsPerMinute,
ProgressUpdatesPerMinute: DefaultProgressUpdatesPerMinute,
MetadataRequestsPerMinute: DefaultMetadataRequestsPerMinute,
}
}
// rateLimitForRequestType returns the configured per-minute limit for a given
// request type, for use in X-RateLimit-* headers.
func (m *DeviceAuthMiddleware) rateLimitForRequestType(requestType string, config DeviceRateLimitConfig) int {
switch requestType {
case "progress":
return config.ProgressUpdatesPerMinute
case "metadata":
return config.MetadataRequestsPerMinute
default: // "sync" and any unknown type
return config.SyncRequestsPerMinute
}
}
func (m *DeviceAuthMiddleware) Authenticate(next echo.HandlerFunc) echo.HandlerFunc {
return func(c *echo.Context) error {
var device database.Devices
@@ -115,15 +151,12 @@ func (m *DeviceAuthMiddleware) Authenticate(next echo.HandlerFunc) echo.HandlerF
deviceUUID := uuid.UUID(device.ID.Bytes)
deviceID := deviceUUID.String()
config := DeviceRateLimitConfig{
SyncRequestsPerMinute: 60,
ProgressUpdatesPerMinute: 120,
MetadataRequestsPerMinute: 30,
}
config := m.rateLimitConfig()
limitForType := m.rateLimitForRequestType(requestType, config)
if !m.rateLimiter.CheckRateLimit(deviceID, requestType, config) {
remaining := m.rateLimiter.GetRemainingRequests(deviceID, requestType, config)
c.Response().Header().Set("X-RateLimit-Limit", "60")
c.Response().Header().Set("X-RateLimit-Limit", strconv.Itoa(limitForType))
c.Response().Header().Set("X-RateLimit-Remaining", strconv.Itoa(remaining))
c.Response().Header().Set("X-RateLimit-Reset", "60")
return c.JSON(http.StatusTooManyRequests, map[string]string{
@@ -134,7 +167,7 @@ func (m *DeviceAuthMiddleware) Authenticate(next echo.HandlerFunc) echo.HandlerF
}
remaining := m.rateLimiter.GetRemainingRequests(deviceID, requestType, config)
c.Response().Header().Set("X-RateLimit-Limit", "60")
c.Response().Header().Set("X-RateLimit-Limit", strconv.Itoa(limitForType))
c.Response().Header().Set("X-RateLimit-Remaining", strconv.Itoa(remaining))
ctx := DeviceContext{
+101 -51
View File
@@ -1,96 +1,146 @@
package middleware
import (
"bookhoard/internal/database"
"fmt"
"regexp"
"sync"
"github.com/go-playground/validator/v10"
)
// PasswordValidator validates password complexity requirements
type PasswordValidator struct{}
// specialCharRegex matches the historical "special character" set used by the
// password complexity rules.
const specialCharRegex = `[!@#$%^&*()_+\-=\[\]{};':"\\|,.<>\/?]`
// Validate checks if a password meets complexity requirements:
// - Minimum 8 characters
// - At least one uppercase letter
// - At least one lowercase letter
// - At least one number
// - At least one special character
// PasswordValidator validates password complexity against the configured rules.
// When a database.SettingsRegistry is wired via SetSettings, rules are read live and the
// regex set is recompiled under a mutex on each validation. Without a registry
// the historical hardcoded defaults (8+ chars, upper/lower/number/special) apply.
type PasswordValidator struct {
settings *database.SettingsRegistry
}
// SetSettings wires the tunable settings registry.
func (v *PasswordValidator) SetSettings(s *database.SettingsRegistry) { v.settings = s }
// compileSpecialRegex isolates the regexp compile (which is safe to call
// concurrently, but we keep it behind a cached var for the no-registry path).
var (
specialOnce sync.Once
specialRe *regexp.Regexp
)
func specialRegex() *regexp.Regexp {
specialOnce.Do(func() {
specialRe = regexp.MustCompile(specialCharRegex)
})
return specialRe
}
func (v *PasswordValidator) rules() database.PasswordRules {
if v.settings != nil {
return v.settings.PasswordRules()
}
return database.PasswordRules{MinLength: 8, Upper: true, Lower: true, Number: true, Special: true}
}
// Validate checks if a password meets the configured complexity requirements.
func (v *PasswordValidator) Validate(fl validator.FieldLevel) bool {
password := fl.Field().String()
return v.CheckPassword(fl.Field().String())
}
// Check minimum length
if len(password) < 8 {
// CheckPassword applies the active rules to a single password.
func (v *PasswordValidator) CheckPassword(password string) bool {
r := v.rules()
if len(password) < r.MinLength {
return false
}
// Check for uppercase
hasUpper := regexp.MustCompile(`[A-Z]`).MatchString(password)
if !hasUpper {
if r.Upper && !regexp.MustCompile(`[A-Z]`).MatchString(password) {
return false
}
// Check for lowercase
hasLower := regexp.MustCompile(`[a-z]`).MatchString(password)
if !hasLower {
if r.Lower && !regexp.MustCompile(`[a-z]`).MatchString(password) {
return false
}
// Check for number
hasNumber := regexp.MustCompile(`[0-9]`).MatchString(password)
if !hasNumber {
if r.Number && !regexp.MustCompile(`[0-9]`).MatchString(password) {
return false
}
// Check for special character
hasSpecial := regexp.MustCompile(`[!@#$%^&*()_+\-=\[\]{};':"\\|,.<>\/?]`).MatchString(password)
if !hasSpecial {
if r.Special && !specialRegex().MatchString(password) {
return false
}
return true
}
// GetPasswordRequirements returns a human-readable list of password requirements
// GetPasswordRequirements returns a human-readable list of the active password
// requirements, driven by the configured rules when a registry is wired.
func GetPasswordRequirements() []string {
return []string{
"At least 8 characters long",
"At least one uppercase letter (A-Z)",
"At least one lowercase letter (a-z)",
"At least one number (0-9)",
"At least one special character (!@#$%^&*()_+-=[]{}|;':\",./<>?)",
}
return defaultPasswordValidator.Requirements()
}
// ValidatePassword checks a password and returns an error if it doesn't meet requirements
func ValidatePassword(password string) error {
if len(password) < 8 {
return fmt.Errorf("password must be at least 8 characters long")
// Requirements returns the human-readable list for the receiver's active rules.
func (v *PasswordValidator) Requirements() []string {
r := v.rules()
var out []string
out = append(out, fmt.Sprintf("At least %d characters long", r.MinLength))
if r.Upper {
out = append(out, "At least one uppercase letter (A-Z)")
}
if r.Lower {
out = append(out, "At least one lowercase letter (a-z)")
}
if r.Number {
out = append(out, "At least one number (0-9)")
}
if r.Special {
out = append(out, "At least one special character (!@#$%^&*()_+-=[]{}|;':\",./<>?)")
}
return out
}
if !regexp.MustCompile(`[A-Z]`).MatchString(password) {
// ValidatePassword checks a password against the default (hardcoded) rules and
// returns an error describing the first unmet requirement. Retained for callers
// that don't have access to a configured PasswordValidator instance.
func ValidatePassword(password string) error {
v := defaultPasswordValidator
r := v.rules()
if len(password) < r.MinLength {
return fmt.Errorf("password must be at least %d characters long", r.MinLength)
}
if r.Upper && !regexp.MustCompile(`[A-Z]`).MatchString(password) {
return fmt.Errorf("password must contain at least one uppercase letter")
}
if !regexp.MustCompile(`[a-z]`).MatchString(password) {
if r.Lower && !regexp.MustCompile(`[a-z]`).MatchString(password) {
return fmt.Errorf("password must contain at least one lowercase letter")
}
if !regexp.MustCompile(`[0-9]`).MatchString(password) {
if r.Number && !regexp.MustCompile(`[0-9]`).MatchString(password) {
return fmt.Errorf("password must contain at least one number")
}
if !regexp.MustCompile(`[!@#$%^&*()_+\-=\[\]{};':"\\|,.<>\/?]`).MatchString(password) {
if r.Special && !specialRegex().MatchString(password) {
return fmt.Errorf("password must contain at least one special character")
}
return nil
}
// RegisterPasswordValidation registers the password validator with the validator instance
// defaultPasswordValidator is used by the package-level helpers
// (GetPasswordRequirements, ValidatePassword) and as the fallback inside
// RegisterPasswordValidation when no registry has been wired. Callers that want
// live rule updates should construct their own PasswordValidator and call
// SetSettings.
var defaultPasswordValidator = &PasswordValidator{}
// RegisterPasswordValidation registers the password validator with the
// validator instance. The registered func re-evaluates rules on every call, so
// changes to the wired registry take effect immediately.
func RegisterPasswordValidation(v *validator.Validate) error {
return v.RegisterValidation("passwordcomplex", func(fl validator.FieldLevel) bool {
pv := &PasswordValidator{}
return pv.Validate(fl)
return defaultPasswordValidator.CheckPassword(fl.Field().String())
})
}
// SetDefaultPasswordSettings wires the settings registry into the package-level
// default validator so that the struct-tag validator (used by echo's
// CustomValidator) and ValidatePassword follow live configuration. Intended to
// be called once at startup.
func SetDefaultPasswordSettings(s *database.SettingsRegistry) {
defaultPasswordValidator.SetSettings(s)
}
+89 -17
View File
@@ -9,15 +9,19 @@ import (
// OPDS 1.2 Feed Structures
type Feed struct {
XMLName xml.Name `xml:"feed"`
Xmlns string `xml:"xmlns,attr"`
OpdsNS string `xml:"xmlns:opds,attr"`
DcNS string `xml:"xmlns:dc,attr"`
ID string `xml:"id"`
Title string `xml:"title"`
Updated string `xml:"updated"`
Links []Link `xml:"link"`
Entries []Entry `xml:"entry"`
XMLName xml.Name `xml:"feed"`
Xmlns string `xml:"xmlns,attr"`
OpdsNS string `xml:"xmlns:opds,attr"`
DcNS string `xml:"xmlns:dc,attr"`
OpenSearchNS string `xml:"xmlns:opensearch,attr,omitempty"`
ID string `xml:"id"`
Title string `xml:"title"`
Updated string `xml:"updated"`
Links []Link `xml:"link"`
TotalResults *int `xml:"opensearch:totalResults,omitempty"`
ItemsPerPage *int `xml:"opensearch:itemsPerPage,omitempty"`
StartIndex *int `xml:"opensearch:startIndex,omitempty"`
Entries []Entry `xml:"entry"`
}
type Entry struct {
@@ -64,17 +68,29 @@ type Category struct {
func NewFeed(feedID, title string) *Feed {
now := time.Now().Format(time.RFC3339)
return &Feed{
Xmlns: "http://www.w3.org/2005/Atom",
OpdsNS: "http://opds-spec.org/2010/",
DcNS: "http://purl.org/dc/elements/1.1/",
ID: feedID,
Title: title,
Updated: now,
Links: []Link{},
Entries: []Entry{},
Xmlns: "http://www.w3.org/2005/Atom",
OpdsNS: "http://opds-spec.org/2010/",
DcNS: "http://purl.org/dc/elements/1.1/",
OpenSearchNS: "http://a9.com/-/spec/opensearch/1.1/",
ID: feedID,
Title: title,
Updated: now,
Links: []Link{},
Entries: []Entry{},
}
}
// SetPagination populates the OpenSearch paging metadata (totalResults,
// itemsPerPage, startIndex). startIndex is 1-based to match the page model.
func (f *Feed) SetPagination(totalResults, itemsPerPage, startIndex int) {
tr := totalResults
ipp := itemsPerPage
si := startIndex
f.TotalResults = &tr
f.ItemsPerPage = &ipp
f.StartIndex = &si
}
// AddLink adds a link to the feed
func (f *Feed) AddLink(href, linkType, rel string) {
f.Links = append(f.Links, Link{
@@ -178,6 +194,62 @@ func (f *Feed) GenerateXMLString() (string, error) {
return xml.Header + string(output), nil
}
// OpenSearchUrl is a single <Url> element in an OpenSearch description.
type OpenSearchUrl struct {
XMLName xml.Name `xml:"Url"`
Type string `xml:"type,attr"`
Template string `xml:"template,attr"`
}
// OpenSearchDescription is an OpenSearch description document used by OPDS
// clients (e.g. KOReader) to discover how to perform catalog searches. Clients
// fetch this document at the catalog's rel="search" link, then substitute
// {searchTerms} in the Url template to execute a query.
type OpenSearchDescription struct {
XMLName xml.Name `xml:"OpenSearchDescription"`
Xmlns string `xml:"xmlns,attr"`
ShortName string `xml:"ShortName"`
Description string `xml:"Description"`
InputEncoding string `xml:"InputEncoding"`
OutputEncoding string `xml:"OutputEncoding"`
Url OpenSearchUrl `xml:"Url"`
}
// NewSearchDescription creates an OpenSearch description document whose Url
// template points clients back to the search results endpoint. The template
// must contain the {searchTerms} placeholder.
func NewSearchDescription(shortName, description, template string) *OpenSearchDescription {
return &OpenSearchDescription{
Xmlns: "http://a9.com/-/spec/opensearch/1.1/",
ShortName: shortName,
Description: description,
InputEncoding: "UTF-8",
OutputEncoding: "UTF-8",
Url: OpenSearchUrl{
Type: "application/atom+xml;profile=opds-catalog;kind=acquisition",
Template: template,
},
}
}
// GenerateXML generates the OpenSearch description XML
func (d *OpenSearchDescription) GenerateXML() ([]byte, error) {
output, err := xml.MarshalIndent(d, "", " ")
if err != nil {
return nil, fmt.Errorf("failed to marshal OpenSearch description: %w", err)
}
return output, nil
}
// GenerateXMLString generates the OpenSearch description XML as a string
func (d *OpenSearchDescription) GenerateXMLString() (string, error) {
output, err := d.GenerateXML()
if err != nil {
return "", err
}
return xml.Header + string(output), nil
}
// NewErrorFeed creates an error feed
func NewErrorFeed(message string) *Feed {
feed := NewFeed(
+105 -4
View File
@@ -71,8 +71,8 @@ func TestNewEntry(t *testing.T) {
t.Errorf("expected Title to be 'Test Title', got '%s'", entry.Title)
}
if entry.Creator != "Test Author" {
t.Errorf("expected Creator to be 'Test Author', got '%s'", entry.Creator)
if entry.Author == nil || entry.Author.Name != "Test Author" {
t.Errorf("expected Author.Name to be 'Test Author', got %v", entry.Author)
}
if entry.Updated != "2023-01-01T00:00:00Z" {
@@ -201,8 +201,8 @@ func TestFeedGenerateXML(t *testing.T) {
`<title>Test Feed</title>`,
`<entry>`,
`<id>urn:uuid:book-id</id>`,
`<dc:title>Test Book</dc:title>`,
`<dc:creator>Test Author</dc:creator>`,
`<title>Test Book</title>`,
`<name>Test Author</name>`,
`<link href="http://example.com/book.epub"`,
`rel="http://opds-spec.org/acquisition/open-access"`,
`<dc:identifier id="bookhoard">book-uuid-123</dc:identifier>`,
@@ -232,6 +232,107 @@ func TestNewErrorFeed(t *testing.T) {
}
}
func TestFeedSetPagination(t *testing.T) {
feed := NewFeed("urn:uuid:test-id", "Test Feed")
feed.SetPagination(1814, 50, 51)
if feed.TotalResults == nil || *feed.TotalResults != 1814 {
t.Errorf("expected TotalResults to be 1814, got %v", feed.TotalResults)
}
if feed.ItemsPerPage == nil || *feed.ItemsPerPage != 50 {
t.Errorf("expected ItemsPerPage to be 50, got %v", feed.ItemsPerPage)
}
if feed.StartIndex == nil || *feed.StartIndex != 51 {
t.Errorf("expected StartIndex to be 51, got %v", feed.StartIndex)
}
}
func TestFeedGenerateXMLPagination(t *testing.T) {
feed := NewFeed("urn:uuid:test-id", "Test Feed")
feed.AddLink("http://example.com/catalog?page=1", "application/atom+xml", "first")
feed.AddLink("http://example.com/catalog?page=1", "application/atom+xml", "previous")
feed.AddLink("http://example.com/catalog?page=2", "application/atom+xml", "self")
feed.AddLink("http://example.com/catalog?page=3", "application/atom+xml", "next")
feed.AddLink("http://example.com/catalog?page=37", "application/atom+xml", "last")
feed.SetPagination(1814, 50, 51)
output, err := feed.GenerateXML()
if err != nil {
t.Fatalf("failed to generate XML: %v", err)
}
outputStr := string(output)
requiredStrings := []string{
`xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/"`,
`<opensearch:totalResults>1814</opensearch:totalResults>`,
`<opensearch:itemsPerPage>50</opensearch:itemsPerPage>`,
`<opensearch:startIndex>51</opensearch:startIndex>`,
`rel="first"`,
`rel="previous"`,
`rel="next"`,
`rel="last"`,
`page=3`,
}
for _, required := range requiredStrings {
if !contains(outputStr, required) {
t.Errorf("generated XML missing required string: %s", required)
}
}
}
func TestFeedGenerateXMLOmitsPaginationWhenUnset(t *testing.T) {
feed := NewFeed("urn:uuid:test-id", "Test Feed")
output, err := feed.GenerateXML()
if err != nil {
t.Fatalf("failed to generate XML: %v", err)
}
outputStr := string(output)
if contains(outputStr, "opensearch:totalResults") {
t.Errorf("expected no totalResults when pagination unset, but found it")
}
if contains(outputStr, "opensearch:itemsPerPage") {
t.Errorf("expected no itemsPerPage when pagination unset, but found it")
}
}
func TestNewSearchDescription(t *testing.T) {
template := "http://example.com/opds/devices/abc/search?q={searchTerms}&token=xyz"
desc := NewSearchDescription("Bookhoard", "Search the library", template)
if desc.ShortName != "Bookhoard" {
t.Errorf("expected ShortName 'Bookhoard', got '%s'", desc.ShortName)
}
if desc.Url.Template != template {
t.Errorf("expected template '%s', got '%s'", template, desc.Url.Template)
}
}
func TestSearchDescriptionGenerateXML(t *testing.T) {
template := "http://example.com/opds/devices/abc/search?q={searchTerms}"
desc := NewSearchDescription("Bookhoard", "Search the library", template)
output, err := desc.GenerateXMLString()
if err != nil {
t.Fatalf("failed to generate XML: %v", err)
}
requiredStrings := []string{
`<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">`,
`<ShortName>Bookhoard</ShortName>`,
`<Url type="application/atom+xml;profile=opds-catalog;kind=acquisition"`,
`template="http://example.com/opds/devices/abc/search?q={searchTerms}"`,
}
for _, required := range requiredStrings {
if !contains(output, required) {
t.Errorf("generated OpenSearch XML missing required string: %s", required)
}
}
}
func contains(s, substr string) bool {
return len(s) >= len(substr) && indexOf(s, substr) >= 0
}
+447
View File
@@ -0,0 +1,447 @@
package router
import (
"bookhoard/internal/database"
"bookhoard/internal/handlers"
"bookhoard/templates"
"bytes"
"context"
"fmt"
"log"
"net/http"
"os"
"path/filepath"
"strconv"
"strings"
"github.com/google/uuid"
"github.com/jackc/pgx/v5/pgtype"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/labstack/echo/v5"
)
func registerAdminLibraryRoutes(cfg *Config, frontendProtected *echo.Group) {
g := frontendProtected.Group("", handlers.AdminMiddleware)
// HTMX: Create library
g.POST("/admin/library/create", func(c *echo.Context) error {
user := c.Get("user").(database.Users)
name := c.FormValue("name")
desc := c.FormValue("description")
libType := c.FormValue("type")
if name == "" || libType == "" {
return c.HTML(http.StatusBadRequest, `<div class="text-sm" style="color: var(--status-danger);">Name and type are required</div>`)
}
_, err := cfg.LibraryService.CreateLibrary(
c.Request().Context(),
name,
desc,
libType,
user.ID,
)
if err != nil {
return c.HTML(http.StatusInternalServerError, `<div class="text-sm" style="color: var(--status-danger);">Failed to create library</div>`)
}
return renderLibraryList(c, cfg)
})
// HTMX: Update library
g.PUT("/admin/library/:id", func(c *echo.Context) error {
libraryID, err := parseAdminUUID(c.Param("id"))
if err != nil {
return c.HTML(http.StatusBadRequest, `<div class="text-sm" style="color: var(--status-danger);">Invalid library ID</div>`)
}
name := c.FormValue("name")
desc := c.FormValue("description")
if name == "" {
return c.HTML(http.StatusBadRequest, `<div class="text-sm" style="color: var(--status-danger);">Name is required</div>`)
}
_, err = cfg.LibraryService.UpdateLibrary(c.Request().Context(), libraryID, name, desc)
if err != nil {
return c.HTML(http.StatusInternalServerError, `<div class="text-sm" style="color: var(--status-danger);">Failed to update library</div>`)
}
return renderLibraryList(c, cfg)
})
// HTMX: Delete library
g.DELETE("/admin/library/:id", func(c *echo.Context) error {
libraryID, err := parseAdminUUID(c.Param("id"))
if err != nil {
return c.HTML(http.StatusBadRequest, `<div class="text-sm" style="color: var(--status-danger);">Invalid library ID</div>`)
}
err = cfg.LibraryService.DeleteLibrary(c.Request().Context(), libraryID)
if err != nil {
return c.HTML(http.StatusInternalServerError, `<div class="text-sm" style="color: var(--status-danger);">Failed to delete library</div>`)
}
return renderLibraryList(c, cfg)
})
// HTMX: Library expanded panel
g.GET("/admin/library/:id/panel", func(c *echo.Context) error {
libraryID, err := parseAdminUUID(c.Param("id"))
if err != nil {
return c.HTML(http.StatusBadRequest, `<div class="text-sm" style="color: var(--status-danger);">Invalid library ID</div>`)
}
return renderLibraryPanel(c, cfg, libraryID)
})
// HTMX: Add folder
g.POST("/admin/library/:id/folders", func(c *echo.Context) error {
libraryID, err := parseAdminUUID(c.Param("id"))
if err != nil {
return c.HTML(http.StatusBadRequest, `<div class="text-sm" style="color: var(--status-danger);">Invalid library ID</div>`)
}
folderPath := c.FormValue("folder_path")
if folderPath == "" {
return renderLibraryPanel(c, cfg, libraryID)
}
if strings.Contains(folderPath, "..") {
return renderLibraryPanelWithError(c, cfg, libraryID, "Path traversal not allowed")
}
cleanPath := filepath.Clean(folderPath)
fileInfo, err := os.Stat(cleanPath)
if err != nil {
return renderLibraryPanelWithError(c, cfg, libraryID, "Folder path does not exist")
}
if !fileInfo.IsDir() {
return renderLibraryPanelWithError(c, cfg, libraryID, "Path must be a directory")
}
_, err = cfg.LibraryService.AddLibraryFolder(c.Request().Context(), libraryID, cleanPath)
if err != nil {
return renderLibraryPanelWithError(c, cfg, libraryID, "Failed to add folder: "+err.Error())
}
return renderLibraryPanel(c, cfg, libraryID)
})
// HTMX: Remove folder
g.DELETE("/admin/library/:id/folders", func(c *echo.Context) error {
libraryID, err := parseAdminUUID(c.Param("id"))
if err != nil {
return c.HTML(http.StatusBadRequest, `<div class="text-sm" style="color: var(--status-danger);">Invalid library ID</div>`)
}
folderPath := c.FormValue("folder_path")
if folderPath == "" {
return renderLibraryPanel(c, cfg, libraryID)
}
err = cfg.LibraryService.DeleteLibraryFolder(c.Request().Context(), libraryID, folderPath)
if err != nil {
return renderLibraryPanelWithError(c, cfg, libraryID, "Failed to remove folder")
}
return renderLibraryPanel(c, cfg, libraryID)
})
// HTMX: Folder browser
g.GET("/admin/library/browse", func(c *echo.Context) error {
path := c.QueryParam("path")
if path == "" {
path = "/"
}
targetInput := c.QueryParam("target_input")
libraryID := c.QueryParam("library_id")
dirs, currentPath, parentPath, err := cfg.LibraryService.BrowseDirectories(c.Request().Context(), path)
if err != nil {
return c.HTML(http.StatusBadRequest, `<div class="text-sm" style="color: var(--status-danger);">Cannot browse: `+err.Error()+`</div>`)
}
entries := make([]templates.DirEntry, len(dirs))
for i, d := range dirs {
fullPath := filepath.Join(currentPath, d)
entries[i] = templates.DirEntry{Name: d, Path: fullPath}
}
var buf bytes.Buffer
err = templates.FolderBrowserContent(currentPath, parentPath, entries, targetInput, libraryID).Render(c.Request().Context(), &buf)
if err != nil {
return err
}
return c.HTML(http.StatusOK, buf.String())
})
// HTMX: Set user visibility for library
g.POST("/admin/library/:id/visibility", func(c *echo.Context) error {
libraryID, err := parseAdminUUID(c.Param("id"))
if err != nil {
return c.HTML(http.StatusBadRequest, `<div class="text-sm" style="color: var(--status-danger);">Invalid library ID</div>`)
}
userIDStr := c.FormValue("user_id")
isVisible := c.FormValue("is_visible") == "true"
userID, err := parseAdminUUID(userIDStr)
if err != nil {
return c.HTML(http.StatusBadRequest, `<div class="text-sm" style="color: var(--status-danger);">Invalid user ID</div>`)
}
_, err = cfg.LibraryService.SetLibraryVisibility(c.Request().Context(), userID, libraryID, isVisible)
if err != nil {
return c.HTML(http.StatusInternalServerError, `<div class="text-sm" style="color: var(--status-danger);">Failed to update visibility</div>`)
}
return renderLibraryPanel(c, cfg, libraryID)
})
}
// renderLibraryList fetches all libraries + users and renders the LibraryList partial.
func renderLibraryList(c *echo.Context, cfg *Config) error {
libraries, err := cfg.LibraryHandler.ListLibrariesData(c.Request().Context())
if err != nil {
return c.HTML(http.StatusInternalServerError, `<div class="text-sm" style="color: var(--status-danger);">Failed to load libraries</div>`)
}
libData := make([]templates.LibraryData, len(libraries))
for i, lib := range libraries {
libUUID, _ := uuid.FromBytes(lib.ID.Bytes[0:16])
folderCount := getFolderCount(c.Request().Context(), cfg, lib.ID)
libData[i] = templates.LibraryData{
ID: libUUID.String(),
Name: lib.Name,
Description: getText(lib.Description),
TypeName: lib.TypeName,
TypeValue: lib.TypeName,
FolderCount: folderCount,
}
}
users, err := cfg.Queries.ListUsers(c.Request().Context())
if err != nil {
log.Printf("ListUsers failed: %v", err)
users = []database.ListUsersRow{}
}
userData := make([]templates.User, len(users))
for i, u := range users {
userUUID, _ := uuid.FromBytes(u.ID.Bytes[0:16])
userData[i] = templates.User{
ID: userUUID.String(),
Username: u.Username,
Email: u.Email,
Role: u.Role,
}
}
var buf bytes.Buffer
err = templates.LibraryList(templates.User{}, libData, userData).Render(c.Request().Context(), &buf)
if err != nil {
return err
}
return c.HTML(http.StatusOK, buf.String())
}
// renderLibraryPanel fetches library details and renders the LibraryPanel partial.
func renderLibraryPanel(c *echo.Context, cfg *Config, libraryID pgtype.UUID) error {
return renderLibraryPanelWithError(c, cfg, libraryID, "")
}
func renderLibraryPanelWithError(c *echo.Context, cfg *Config, libraryID pgtype.UUID, errMsg string) error {
ctx := c.Request().Context()
libraryIDStr := uuid.UUID(libraryID.Bytes).String()
// Get library details
lib, err := cfg.LibraryService.GetLibrary(ctx, libraryID)
if err != nil {
return c.HTML(http.StatusNotFound, `<div class="text-sm" style="color: var(--status-danger);">Library not found</div>`)
}
libData := templates.LibraryData{
ID: libraryIDStr,
Name: lib.Name,
Description: getText(lib.Description),
TypeName: lib.TypeName,
TypeValue: lib.TypeName,
}
// Get folders
dbFolders, err := cfg.LibraryService.GetLibraryFolders(ctx, libraryID)
if err != nil {
log.Printf("GetLibraryFolders failed: %v", err)
}
folders := make([]templates.FolderData, len(dbFolders))
for i, f := range dbFolders {
folders[i] = templates.FolderData{FolderPath: f.FolderPath}
}
libData.FolderCount = len(folders)
// Get users
dbUsers, err := cfg.Queries.ListUsers(ctx)
if err != nil {
log.Printf("ListUsers failed: %v", err)
dbUsers = []database.ListUsersRow{}
}
userData := make([]templates.User, len(dbUsers))
for i, u := range dbUsers {
userUUID, _ := uuid.FromBytes(u.ID.Bytes[0:16])
userData[i] = templates.User{
ID: userUUID.String(),
Username: u.Username,
Email: u.Email,
}
}
// Get visibility for all users
visibility := make([]templates.UserVisibilityData, len(userData))
for i, u := range userData {
userUUID, _ := parseAdminUUID(u.ID)
visibleLibs, err := cfg.LibraryService.GetUserVisibleLibraries(ctx, userUUID)
if err != nil {
log.Printf("GetUserVisibleLibraries failed: %v", err)
}
isVisible := false
for _, vl := range visibleLibs {
if vl.ID.Bytes == libraryID.Bytes {
isVisible = true
break
}
}
visibility[i] = templates.UserVisibilityData{
UserID: u.ID,
Username: u.Username,
Email: u.Email,
IsVisible: isVisible,
}
}
// Get issue count
issueStats, err := cfg.ProcessingIssuesHandler.GetProcessingIssueStatsData(ctx, libraryID)
if err != nil {
log.Printf("GetProcessingIssueStats failed: %v", err)
}
issueCount := issueStats.ErrorCount + issueStats.WarningCount + issueStats.InfoCount
// Get current user for template
tmplUser := templates.User{}
if u, ok := c.Get("user").(database.Users); ok {
userUUID, _ := uuid.FromBytes(u.ID.Bytes[0:16])
tmplUser = templates.User{
ID: userUUID.String(),
Username: u.Username,
Role: u.Role,
}
}
var buf bytes.Buffer
err = templates.LibraryPanel(tmplUser, libraryIDStr, libData, folders, userData, visibility, int(issueCount)).Render(ctx, &buf)
if err != nil {
return err
}
html := buf.String()
if errMsg != "" {
html = `<div class="p-3 mb-3 rounded-lg text-sm" style="background-color: color-mix(in srgb, var(--status-danger) 12%, var(--bg-secondary)); color: var(--status-danger);">` + errMsg + `</div>` + html
}
return c.HTML(http.StatusOK, html)
}
func getFolderCount(ctx context.Context, cfg *Config, libraryID pgtype.UUID) int {
folders, err := cfg.LibraryService.GetLibraryFolders(ctx, libraryID)
if err != nil {
return 0
}
return len(folders)
}
func parseAdminUUID(s string) (pgtype.UUID, error) {
parsed, err := uuid.Parse(s)
if err != nil {
return pgtype.UUID{}, err
}
return pgtype.UUID{Bytes: parsed, Valid: true}, nil
}
func getAdminStats(ctx context.Context, cfg *Config) templates.AdminStats {
stats := templates.AdminStats{}
libs, _ := cfg.LibraryHandler.ListLibrariesData(ctx)
stats.LibraryCount = len(libs)
users, _ := cfg.Queries.ListUsers(ctx)
stats.UserCount = len(users)
if pool, ok := cfg.DBPool.(*pgxpool.Pool); ok {
_ = pool.QueryRow(ctx, "SELECT COUNT(*) FROM media_items").Scan(&stats.MediaCount)
_ = pool.QueryRow(ctx, "SELECT COUNT(*) FROM devices").Scan(&stats.DeviceCount)
}
return stats
}
func registerAdminSettingsRoutes(cfg *Config, frontendProtected *echo.Group) {
g := frontendProtected.Group("", handlers.AdminMiddleware)
g.PUT("/admin/settings/scan", func(c *echo.Context) error {
ctx := c.Request().Context()
autoScan := c.FormValue("auto_scan_enabled") == "true"
intervalStr := c.FormValue("scan_poll_interval_seconds")
interval, err := strconv.Atoi(intervalStr)
if err != nil || interval < 1 || interval > 3600 {
return c.HTML(http.StatusBadRequest, `<div class="text-sm" style="color: var(--status-danger);">Interval must be between 1 and 3600 seconds</div>`)
}
autoScanStr := "false"
if autoScan {
autoScanStr = "true"
}
_ = cfg.Queries.UpdateSystemSetting(ctx, database.UpdateSystemSettingParams{
SettingKey: "auto_scan_enabled",
SettingValue: autoScanStr,
})
_ = cfg.Queries.UpdateSystemSetting(ctx, database.UpdateSystemSettingParams{
SettingKey: "scan_poll_interval_seconds",
SettingValue: strconv.Itoa(interval),
})
// Refresh the registry cache so the change is visible immediately.
if cfg.Settings != nil {
cfg.Settings.Reload(ctx)
}
scanSettings := templates.ScanSettingsData{
AutoScanEnabled: autoScan,
ScanPollIntervalSeconds: interval,
}
var buf bytes.Buffer
_ = templates.ScanSettingsSection(scanSettings).Render(ctx, &buf)
return c.HTML(http.StatusOK, buf.String())
})
// HTMX endpoint for saving a single tunable setting. Returns a small HTML
// status snippet rendered into the row's status span.
g.PUT("/admin/settings/tunable", func(c *echo.Context) error {
ctx := c.Request().Context()
key := c.FormValue("key")
value := c.FormValue("value")
if cfg.SystemSettingsHandler == nil {
return c.HTML(http.StatusServiceUnavailable, `<span style="color: var(--status-danger);">settings unavailable</span>`)
}
resp, err := cfg.SystemSettingsHandler.ApplySetting(ctx, key, value)
if err != nil {
return c.HTML(http.StatusBadRequest, fmt.Sprintf(`<span style="color: var(--status-danger);">%s</span>`, err.Error()))
}
color := "var(--status-success)"
msg := "Saved"
if resp.ReloadRequired {
color = "var(--status-warning)"
msg = "Saved — restart required"
}
return c.HTML(http.StatusOK, fmt.Sprintf(`<span style="color: %s;">%s</span>`, color, msg))
})
}
+143 -25
View File
@@ -4,12 +4,12 @@ import (
"bytes"
"context"
"encoding/json"
"fmt"
"log"
"net/http"
"strconv"
"time"
"bookhoard/internal/config"
"bookhoard/internal/database"
"bookhoard/internal/handlers"
"bookhoard/internal/services"
@@ -215,6 +215,9 @@ func registerFrontendRoutes(cfg *Config) {
bookInfoList = []handlers.BookInfo{}
}
seriesUserUUID, _ := uuid.Parse(user.ID)
bookInfoList = handlers.MarkActiveConflicts(c.Request().Context(), cfg.Queries, pgtype.UUID{Bytes: seriesUserUUID, Valid: true}, bookInfoList)
var buf bytes.Buffer
err = templates.BrowseDetail(user, "📚", "Series", seriesName, seriesName, "/series", "All Series", "📚", "This series doesn't have any books yet", bookInfoList, errorMsg).Render(c.Request().Context(), &buf)
if err != nil {
@@ -267,6 +270,9 @@ func registerFrontendRoutes(cfg *Config) {
bookInfoList = []handlers.BookInfo{}
}
tagUserUUID, _ := uuid.Parse(user.ID)
bookInfoList = handlers.MarkActiveConflicts(c.Request().Context(), cfg.Queries, pgtype.UUID{Bytes: tagUserUUID, Valid: true}, bookInfoList)
var buf bytes.Buffer
err = templates.BrowseDetail(user, "🏷️", "Tag", tagName, tagName, "/bookshelf", "Bookshelf", "🏷️", "No books found with this tag", bookInfoList, errorMsg).Render(c.Request().Context(), &buf)
if err != nil {
@@ -354,6 +360,7 @@ func registerFrontendRoutes(cfg *Config) {
}
var buf bytes.Buffer
bookInfoList = handlers.MarkActiveConflicts(c.Request().Context(), cfg.Queries, pgtype.UUID{Bytes: userUUID, Valid: true}, bookInfoList)
err = templates.BookShelf(user, libData, libraryID, errorMsg, savedFilters, bookInfoList, limit, offset, totalCount).Render(c.Request().Context(), &buf)
if err != nil {
return err
@@ -408,7 +415,8 @@ func registerFrontendRoutes(cfg *Config) {
// Get only visible sections for the dashboard display
visibleSections := cfg.DashboardService.FilterHiddenCollections(allSections, prefs.HiddenCollections)
sectionData := handlers.BuildSections(visibleSections, libraryID)
userPgID := pgtype.UUID{Bytes: userUUID, Valid: true}
sectionData := handlers.MarkActiveConflictsSections(c.Request().Context(), cfg.Queries, userPgID, handlers.BuildSections(visibleSections, libraryID))
allSectionsData := handlers.BuildSections(allSections, libraryID)
var buf bytes.Buffer
@@ -623,6 +631,7 @@ func registerFrontendRoutes(cfg *Config) {
Description: collection.Description.String,
Color: collection.Color.String,
Icon: collection.Icon.String,
IsSystem: collection.IsSystemCollection.Bool,
}
var buf bytes.Buffer
@@ -725,10 +734,7 @@ func registerFrontendRoutes(cfg *Config) {
}
// Get base URL from database config with fallback to config/env var
baseURL := config.GetBaseURL(c.Request().Context(), cfg.Queries)
if baseURL == "" {
baseURL = cfg.Cfg.BaseURL
}
baseURL := cfg.getBaseURL(c.Request().Context())
var buf bytes.Buffer
err = templates.Devices(user, devices, pendingList, errorMsg, baseURL).Render(c.Request().Context(), &buf)
@@ -805,8 +811,9 @@ func registerFrontendRoutes(cfg *Config) {
if err != nil {
return renderErrorPage(c, "Error loading user", "user_load_error")
}
stats := getAdminStats(c.Request().Context(), cfg)
var buf bytes.Buffer
err = templates.Admin(user).Render(c.Request().Context(), &buf)
err = templates.Admin(user, stats).Render(c.Request().Context(), &buf)
if err != nil {
return err
}
@@ -818,8 +825,9 @@ func registerFrontendRoutes(cfg *Config) {
if err != nil {
return renderErrorPage(c, "Error loading user", "user_load_error")
}
stats := getAdminStats(c.Request().Context(), cfg)
var buf bytes.Buffer
err = templates.Admin(user).Render(c.Request().Context(), &buf)
err = templates.Admin(user, stats).Render(c.Request().Context(), &buf)
if err != nil {
return err
}
@@ -843,11 +851,14 @@ func registerFrontendRoutes(cfg *Config) {
libData := make([]templates.LibraryData, len(libraries))
for i, lib := range libraries {
libUUID, _ := uuid.FromBytes(lib.ID.Bytes[0:16])
folders, _ := cfg.LibraryService.GetLibraryFolders(c.Request().Context(), lib.ID)
libData[i] = templates.LibraryData{
ID: libUUID.String(),
Name: lib.Name,
Description: getText(lib.Description),
TypeName: lib.TypeName,
TypeValue: lib.TypeName,
FolderCount: len(folders),
}
}
@@ -934,6 +945,67 @@ func registerFrontendRoutes(cfg *Config) {
return c.HTML(http.StatusOK, buf.String())
}))
// Admin hash conflicts page: content-duplicate groups flagged during hash
// backfill or rescan, resolved by keeping all copies or merging into one.
frontendProtected.GET("/admin/hash-conflicts", handlers.AdminMiddleware(func(c *echo.Context) error {
user, err := getTemplateUserWithTheme(c, cfg)
if err != nil {
return renderErrorPage(c, "Error loading user", "user_load_error")
}
pending, err := cfg.Queries.ListPendingHashConflicts(c.Request().Context())
if err != nil {
return renderErrorPage(c, "Error loading hash conflicts", "conflicts_load_error")
}
conflicts := make([]templates.HashConflictData, 0, len(pending))
for _, p := range pending {
conflict := templates.HashConflictData{
ID: uuid.UUID(p.ID.Bytes).String(),
LibraryName: p.LibraryName,
SHA256: p.FileSha256,
SHAShort: p.FileSha256[:16] + "…",
CreatedAt: p.CreatedAt.Time.Format("Jan 2, 2006"),
Items: []templates.HashConflictItemData{},
}
items, err := cfg.Queries.ListMediaItemsBySHA256AndLibrary(c.Request().Context(), database.ListMediaItemsBySHA256AndLibraryParams{
FileSha256: pgtype.Text{String: p.FileSha256, Valid: true},
LibraryID: p.LibraryID,
})
if err != nil {
continue
}
for _, mi := range items {
counts, err := cfg.Queries.GetMediaItemUsageCounts(c.Request().Context(), mi.ID)
if err != nil {
counts = database.GetMediaItemUsageCountsRow{}
}
totalData := counts.ProgressCount + counts.HighlightsCount + counts.BookmarksCount + counts.NotesCount + counts.CollectionsCount
conflict.Items = append(conflict.Items, templates.HashConflictItemData{
ID: uuid.UUID(mi.ID.Bytes).String(),
Title: mi.Title,
Author: mi.Author.String,
FilePath: mi.FilePath,
FileSize: mi.FileSize.Int64,
UsageSummary: fmt.Sprintf("%d progress, %d highlights, %d bookmarks, %d notes, %d collections",
counts.ProgressCount, counts.HighlightsCount, counts.BookmarksCount, counts.NotesCount, counts.CollectionsCount),
HasReadingData: totalData > 0,
})
}
conflicts = append(conflicts, conflict)
}
var buf bytes.Buffer
err = templates.AdminHashConflicts(user, conflicts).Render(c.Request().Context(), &buf)
if err != nil {
return err
}
return c.HTML(http.StatusOK, buf.String())
}))
// Admin users page
frontendProtected.GET("/admin/users", handlers.AdminMiddleware(func(c *echo.Context) error {
user, err := getTemplateUserWithTheme(c, cfg)
@@ -985,24 +1057,67 @@ func registerFrontendRoutes(cfg *Config) {
return renderErrorPage(c, "Error loading user", "user_load_error")
}
ctx := c.Request().Context()
// Fetch current system configuration - just base_url
baseURL := config.GetBaseURL(c.Request().Context(), cfg.Queries)
if baseURL == "" {
baseURL = cfg.Cfg.BaseURL
}
baseURL := cfg.getBaseURL(ctx)
systemConfig := map[string]string{
"base_url": baseURL,
"default_timezone": "UTC",
}
defaultTimezone, err := cfg.Queries.GetSystemTimezone(c.Request().Context())
defaultTimezone, err := cfg.Queries.GetSystemTimezone(ctx)
if err == nil && defaultTimezone != "" {
systemConfig["default_timezone"] = defaultTimezone
}
// Fetch scan settings
scanSettings := templates.ScanSettingsData{
AutoScanEnabled: true,
ScanPollIntervalSeconds: 60,
}
if val, err := cfg.Queries.GetSystemSetting(ctx, "auto_scan_enabled"); err == nil {
scanSettings.AutoScanEnabled = val == "true"
}
if val, err := cfg.Queries.GetSystemSetting(ctx, "scan_poll_interval_seconds"); err == nil {
if n, err := strconv.Atoi(val); err == nil {
scanSettings.ScanPollIntervalSeconds = n
}
}
// Load tunable settings entries from the registry. Exclude keys that
// already have their own dedicated UI cards (timezone dropdown, scan
// settings) so they aren't listed twice.
dedicatedUI := map[string]bool{
"default_timezone": true,
"scan_poll_interval_seconds": true,
"auto_scan_enabled": true,
}
var tunableSettings []templates.SettingEntry
if cfg.Settings != nil {
for _, e := range cfg.Settings.All() {
if dedicatedUI[e.Key] {
continue
}
tunableSettings = append(tunableSettings, templates.SettingEntry{
Key: e.Key,
Value: e.Value,
Type: e.Type,
Min: e.Min,
Max: e.Max,
RequiresRestart: e.RequiresRestart,
Category: e.Category,
Group: e.Group,
Description: e.Description,
IsDefault: e.IsDefault,
})
}
}
liveGroups, restartGroups := templates.GroupTunableSettings(tunableSettings)
var buf bytes.Buffer
err = templates.AdminSettings(user, systemConfig, "").Render(c.Request().Context(), &buf)
err = templates.AdminSettings(user, systemConfig, scanSettings, liveGroups, restartGroups, "").Render(ctx, &buf)
if err != nil {
return err
}
@@ -1044,6 +1159,11 @@ func registerFrontendRoutes(cfg *Config) {
return c.HTML(http.StatusOK, buf.String())
}))
// Admin library HTMX endpoints
registerAdminLibraryRoutes(cfg, frontendProtected)
// Admin settings HTMX endpoints
registerAdminSettingsRoutes(cfg, frontendProtected)
// ============================================================================
// LEGACY API ROUTES (for backward compatibility)
// ============================================================================
@@ -1077,10 +1197,7 @@ func registerFrontendRoutes(cfg *Config) {
}
// Get base URL from database config with fallback to config/env var
baseURL := config.GetBaseURL(c.Request().Context(), cfg.Queries)
if baseURL == "" {
baseURL = cfg.Cfg.BaseURL
}
baseURL := cfg.getBaseURL(c.Request().Context())
var buf bytes.Buffer
err = templates.Devices(user, devices, pendingList, errorMsg, baseURL).Render(c.Request().Context(), &buf)
@@ -1212,13 +1329,14 @@ func registerFrontendRoutes(cfg *Config) {
// Assemble response (no field duplication!)
detail := handlers.MediaDetail{
MediaItems: mediaItem, // Embedded - ALL fields available
Rating: rating,
Collections: collections,
ReadingProgress: progress,
ActiveConflict: activeConflict,
NotesCount: len(notes),
HighlightsCount: len(highlights),
MediaItems: mediaItem, // Embedded - ALL fields available
Rating: rating,
Collections: collections,
ReadingProgress: progress,
ActiveConflict: activeConflict,
NotesCount: len(notes),
HighlightsCount: len(highlights),
DeletedAnnotations: handlers.DeletedAnnotationsForBook(c.Request().Context(), cfg.Queries, pgUserID, pgMediaUUID),
}
// Render template
+12
View File
@@ -118,6 +118,17 @@ func resolveLibrary(c *echo.Context, cfg *Config, userUUID string) LibraryResolu
libraries = []database.GetUserVisibleLibrariesRow{}
}
counts, countErr := cfg.Queries.GetVisibleLibraryMediaCounts(c.Request().Context(), uuidToPGType(userU))
if countErr != nil {
log.Printf("GetVisibleLibraryMediaCounts failed: %v", countErr)
counts = []database.GetVisibleLibraryMediaCountsRow{}
}
countMap := make(map[string]int64, len(counts))
for _, mc := range counts {
mcUUID, _ := uuid.FromBytes(mc.ID.Bytes[0:16])
countMap[mcUUID.String()] = mc.MediaCount
}
res.Libraries = make([]templates.LibraryData, len(libraries))
for i, lib := range libraries {
libUUID, _ := uuid.FromBytes(lib.ID.Bytes[0:16])
@@ -126,6 +137,7 @@ func resolveLibrary(c *echo.Context, cfg *Config, userUUID string) LibraryResolu
Name: lib.Name,
Description: getText(lib.Description),
TypeName: lib.TypeName,
MediaCount: countMap[libUUID.String()],
}
}
+2
View File
@@ -38,6 +38,8 @@ func registerLibraryRoutes(cfg *Config) {
adminLibrary.GET("/:id/stats", cfg.LibraryHandler.GetLibraryStats)
adminLibrary.GET("/:id/issues/list", cfg.ProcessingIssuesHandler.ListProcessingIssues)
adminLibrary.GET("/:id/issues/stats", cfg.ProcessingIssuesHandler.GetProcessingIssueStats)
adminLibrary.POST("/:id/issues/:issueId/:mediaItemId/resolve", cfg.ProcessingIssuesHandler.ResolveProcessingIssue)
adminLibrary.DELETE("/:id/issues/:issueId", cfg.ProcessingIssuesHandler.DeleteProcessingIssue)
adminLibrary.POST("/:id/scan", func(c *echo.Context) error {
libraryID := c.Param("id")
scanReq := map[string]interface{}{
+13
View File
@@ -41,6 +41,19 @@ func registerMediaRoutes(cfg *Config) {
protected.PUT("/media-items/:id/highlights/:highlightId", cfg.MediaHandler.UpdateMediaHighlight)
protected.DELETE("/media-items/:id/highlights/:highlightId", cfg.MediaHandler.DeleteMediaHighlight)
// Bookmark routes (all authenticated users)
protected.GET("/media-items/:id/bookmarks", cfg.MediaHandler.GetMediaBookmarks)
protected.POST("/media-items/:id/bookmarks", cfg.MediaHandler.CreateMediaBookmark)
protected.PUT("/media-items/:id/bookmarks/:bookmarkId", cfg.MediaHandler.UpdateMediaBookmark)
protected.DELETE("/media-items/:id/bookmarks/:bookmarkId", cfg.MediaHandler.DeleteMediaBookmark)
// Deleted-annotation history (all authenticated users): tombstoned
// highlights/notes/bookmarks restorable or permanently removable from the
// book page's "recently deleted" list.
protected.GET("/media-items/:id/annotations/deleted", cfg.MediaHandler.GetDeletedAnnotations)
protected.POST("/media-items/:id/annotations/:annotationId/restore", cfg.MediaHandler.RestoreDeletedAnnotation)
protected.DELETE("/media-items/:id/annotations/:annotationId", cfg.MediaHandler.PurgeDeletedAnnotation)
// Admin-only media routes
admin.POST("/media-items", cfg.MediaHandler.CreateMediaItem)
admin.PUT("/media-items/:id", cfg.MediaHandler.UpdateMediaItem)
+39 -2
View File
@@ -39,6 +39,7 @@ type Config struct {
Echo *echo.Echo
Queries *database.Queries
Cfg *config.Config
Settings *database.SettingsRegistry
DBPool interface{} // pgxpool.Pool interface
AuthHandler *handlers.AuthHandler
LibraryHandler *handlers.LibraryHandler
@@ -46,6 +47,7 @@ type Config struct {
MediaHandler *handlers.MediaHandler
MatchingHandler *handlers.MatchingHandler
ProcessingIssuesHandler *handlers.ProcessingIssuesHandler
HashConflictsHandler *handlers.HashConflictsHandler
KOReaderHandler *handlers.KOReaderHandler
WSHandler *handlers.WSHandler
ConflictHandler *handlers.ConflictHandler
@@ -72,6 +74,24 @@ type Config struct {
LibraryService *services.LibraryService
}
// getBaseURL returns the configured base URL from the database, falling back to
// the env var / config default. Uses a closure to adapt the database query to
// config.SystemConfigGetter.
func (cfg *Config) getBaseURL(ctx context.Context) string {
getter := func(ctx context.Context, key string) (string, error) {
row, err := cfg.Queries.GetSystemConfig(ctx, key)
if err != nil {
return "", err
}
return row.Value, nil
}
baseURL := config.GetBaseURL(ctx, getter)
if baseURL == "" {
baseURL = cfg.Cfg.BaseURL
}
return baseURL
}
// createJWTMiddleware creates a JWT middleware with proper user context setup
func createJWTMiddleware(cfg *Config) echo.MiddlewareFunc {
return echojwt.WithConfig(echojwt.Config{
@@ -193,10 +213,12 @@ func RegisterRoutes(cfg *Config) *handlers.Handler {
// Setup redirect middleware - must run before all routes
e.Pre(setupRedirectMiddleware(cfg))
// Rate limiter
// Rate limiter. The per-minute value comes from the settings registry (DB);
// the enabled flag stays env-driven since disabling rate limiting is a
// deployment-time decision, not a runtime tunable.
rateLimiterConfig := ratelimit.RateLimiterConfig{
Enabled: cfg.Cfg.RateLimitEnabled,
RequestsPerMinute: cfg.Cfg.RequestsPerMinute,
RequestsPerMinute: cfg.Settings.AuthRateLimit(),
CleanupInterval: 5 * time.Minute,
}
rateLimiter := ratelimit.NewRateLimiter(rateLimiterConfig)
@@ -273,6 +295,17 @@ func RegisterRoutes(cfg *Config) *handlers.Handler {
}
}()
// One-time hash backfill: compute and store SHA-256 for media items
// imported before hashing existed, then flag any content-duplicate groups
// for admin review on the Hash Conflicts page. Runs independently of
// auto-scan (it is a one-shot self-heal, not a recurring scan) and is a
// no-op once every item is hashed. Delayed so it does not compete with
// startup scans for disk I/O.
go func() {
time.Sleep(30 * time.Second)
services.NewHashBackfillService(cfg.Queries).Run(context.Background())
}()
// Register progress routes with actual handler
registerProgressRoutes(cfg, scannerHandler)
@@ -280,5 +313,9 @@ func RegisterRoutes(cfg *Config) *handlers.Handler {
admin := protected.Group("", handlers.AdminMiddleware)
registerScannerRoutes(admin, scannerHandler)
// Hash conflict routes (admin only)
admin.GET("/api/admin/hash-conflicts", cfg.HashConflictsHandler.ListHashConflicts)
admin.POST("/api/admin/hash-conflicts/:id/resolve", cfg.HashConflictsHandler.ResolveHashConflict)
return scannerHandler
}
+8 -2
View File
@@ -112,9 +112,15 @@ func handleSearchHTML(c *echo.Context, cfg *Config) error {
CoverImagePath: utils.ResolveMediaURL(pgtype.UUID{Bytes: bookLibUUID, Valid: true}, book.CoverImagePath),
}
}
// Render using BooksGrid template
// Stamp active conflict flags so cards route the play action correctly
bookInfoList = handlers.MarkActiveConflicts(c.Request().Context(), cfg.Queries, user.ID, bookInfoList)
// Render using BooksGrid template (or BookPickerGrid for collection picker)
var buf bytes.Buffer
err = templates.BooksGrid(bookInfoList, limit, offset, totalCount, libraryID).Render(c.Request().Context(), &buf)
if c.QueryParam("show_checkbox") == "true" {
err = templates.BookPickerGrid(bookInfoList).Render(c.Request().Context(), &buf)
} else {
err = templates.BooksGrid(bookInfoList, limit, offset, totalCount, libraryID).Render(c.Request().Context(), &buf)
}
if err != nil {
log.Printf("Template render error: %v", err)
return c.HTML(http.StatusInternalServerError, `<div style="color: red;">Render error</div>`)
+40 -10
View File
@@ -14,7 +14,40 @@ import (
)
func isSetupComplete(cfg *Config) bool {
return setupstatus.IsSetupComplete(context.Background(), cfg.Queries)
getter := func(ctx context.Context) (string, error) {
row, err := cfg.Queries.GetSystemConfig(ctx, "base_url")
if err != nil {
return "", err
}
return row.Value, nil
}
return setupstatus.IsSetupComplete(context.Background(), cfg.Queries, getter)
}
// setupAllowedAPIRoutes lists API endpoints that remain accessible before
// initial setup is complete so the server can be configured via API.
var setupAllowedAPIRoutes = []string{
"/api/auth/register",
"/api/auth/login",
"/api/system/config",
}
// isAllowedDuringSetup reports whether a request path should bypass the setup
// gate. This includes the setup page itself, static assets, health checks, and
// the minimal set of API routes needed to perform initial configuration.
func isAllowedDuringSetup(path string) bool {
if path == "/setup" || path == "/setup/" {
return true
}
if strings.HasPrefix(path, "/static/") || path == "/health" || path == "/favicon.ico" {
return true
}
for _, route := range setupAllowedAPIRoutes {
if path == route || strings.HasPrefix(path, route+"/") {
return true
}
}
return false
}
func setupRedirectMiddleware(cfg *Config) echo.MiddlewareFunc {
@@ -22,19 +55,16 @@ func setupRedirectMiddleware(cfg *Config) echo.MiddlewareFunc {
return func(c *echo.Context) error {
path := c.Request().URL.Path
if path == "/setup" || path == "/setup/" {
return next(c)
}
if strings.HasPrefix(path, "/api/") {
return next(c)
}
if strings.HasPrefix(path, "/static/") || path == "/health" || path == "/favicon.ico" {
if isAllowedDuringSetup(path) {
return next(c)
}
if !isSetupComplete(cfg) {
if strings.HasPrefix(path, "/api/") {
return c.JSON(http.StatusServiceUnavailable, map[string]string{
"error": "Server setup is not complete. Configure an admin account and base_url via the setup wizard or API.",
})
}
return c.Redirect(http.StatusFound, "/setup")
}
+1
View File
@@ -24,6 +24,7 @@ func registerSyncRoutes(cfg *Config) {
// KOReader sync routes (device authentication required)
koreaderSync := e.Group("/api/sync/koreader")
koreaderSync.POST("/progress", cfg.DeviceAuthMiddleware.Authenticate(cfg.KOReaderHandler.SyncProgress))
koreaderSync.GET("/resolve", cfg.DeviceAuthMiddleware.Authenticate(cfg.KOReaderHandler.ResolveBook))
koreaderSync.GET("/metadata/:uuid", cfg.DeviceAuthMiddleware.Authenticate(cfg.KOReaderHandler.GetMetadata))
koreaderSync.GET("/library", cfg.DeviceAuthMiddleware.Authenticate(cfg.KOReaderHandler.GetLibrary))
koreaderSync.POST("/bookmarks", cfg.DeviceAuthMiddleware.Authenticate(cfg.KOReaderHandler.SyncBookmarks))
+6
View File
@@ -17,4 +17,10 @@ func registerSystemRoutes(cfg *Config) {
// System configuration routes (admin-only)
system.GET("/config", cfg.SidecarHandler.GetSystemConfiguration)
system.PUT("/config", cfg.SidecarHandler.UpdateSystemConfiguration)
// Unified tunable settings (admin-only). These back the admin UI's
// editable System Settings sections and supersede the legacy
// /api/libraries/scan-settings JSON routes.
system.GET("/settings", cfg.SystemSettingsHandler.GetSettings)
system.PUT("/settings", cfg.SystemSettingsHandler.UpdateSetting)
}
+15 -19
View File
@@ -46,13 +46,15 @@ type LinkBookRequest struct {
// BookMatchingService handles universal book matching
type BookMatchingService struct {
db *database.Queries
db *database.Queries
resolver *BookResolver
}
// NewBookMatchingService creates a new book matching service
func NewBookMatchingService(db *database.Queries) *BookMatchingService {
return &BookMatchingService{
db: db,
db: db,
resolver: NewBookResolver(db),
}
}
@@ -167,27 +169,21 @@ func (s *BookMatchingService) matchByOPFUUID(ctx context.Context, identifiers []
return nil
}
// matchBySHA256 attempts to match by file SHA-256 hash
// matchBySHA256 attempts to match by file SHA-256 hash.
// Uses the shared BookResolver so it is both indexed (no full-table scan) and
// format-aware: a converted/alternate format hash (media_item_formats) matches
// in addition to the primary media_items.file_sha256.
func (s *BookMatchingService) matchBySHA256(ctx context.Context, sha256 string) *BookMatch {
items, err := s.db.ListMediaItems(ctx, database.ListMediaItemsParams{
Limit: 1000,
Offset: 0,
})
if err != nil {
item, method, err := s.resolver.ResolveBySHA256(ctx, sha256)
if err != nil || !item.ID.Valid {
return nil
}
for _, item := range items {
if item.FileSha256.Valid && item.FileSha256.String == sha256 {
return &BookMatch{
MediaItemID: item.ID.Bytes,
BookhoardUUID: item.ID.Bytes,
Confidence: 0.9,
MatchMethod: "sha256_match",
}
}
return &BookMatch{
MediaItemID: item.ID.Bytes,
BookhoardUUID: item.ID.Bytes,
Confidence: 0.9,
MatchMethod: "sha256_" + string(method),
}
return nil
}
// matchByOPFIdentifier attempts to match by OPF identifier
+68
View File
@@ -0,0 +1,68 @@
package services
import (
"bookhoard/internal/database"
"context"
"errors"
"fmt"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgtype"
)
// ResolveMethod describes how a media item was resolved from a client-supplied identifier.
type ResolveMethod string
const (
MethodNone ResolveMethod = ""
MethodSHA256 ResolveMethod = "sha256" // matched on media_items.file_sha256
MethodSHA256Format ResolveMethod = "sha256_format" // matched on media_item_formats.file_sha256 (converted/alternate format)
)
// BookResolver is the single shared path from a client-supplied identifier to a
// media_item.
//
// All client/sync interfaces (koreader, kobo, OPDS, the device-link UI, and any
// future mobile app) should resolve books through BookResolver so they share
// identical matching semantics. In particular it provides format-aware SHA-256
// matching: a converted file (KEPUB/PDF) whose hash lives in media_item_formats
// resolves just as well as the primary format. The import-time SHA-256 is the
// canonical shared identifier across every client.
type BookResolver struct {
db *database.Queries
}
// NewBookResolver constructs a resolver backed by the given queries.
func NewBookResolver(db *database.Queries) *BookResolver {
return &BookResolver{db: db}
}
// ResolveBySHA256 resolves a media item by its content hash. It checks the
// primary media_items.file_sha256 first, then media_item_formats.file_sha256 so
// that a converted/alternate format (KEPUB, PDF, ...) also matches. Returns the
// matched item and how it matched, or pgx.ErrNoRows when no item has this hash.
func (r *BookResolver) ResolveBySHA256(ctx context.Context, sha256 string) (database.MediaItems, ResolveMethod, error) {
if sha256 == "" {
return database.MediaItems{}, MethodNone, pgx.ErrNoRows
}
sha := pgtype.Text{String: sha256, Valid: true}
// 1. Primary content hash (the file the media item was imported from).
if mi, err := r.db.GetMediaItemBySHA256(ctx, sha); err == nil {
return mi, MethodSHA256, nil
} else if !errors.Is(err, pgx.ErrNoRows) {
return database.MediaItems{}, MethodNone, fmt.Errorf("resolve by sha256 (primary): %w", err)
}
// 2. Per-format hash (a converted/alternate format: KEPUB, PDF, ...).
formatRow, err := r.db.GetMediaItemFormatBySHA256(ctx, sha)
if err == nil {
if mi, err := r.db.GetMediaItem(ctx, formatRow.MediaItemID); err == nil {
return mi, MethodSHA256Format, nil
}
} else if !errors.Is(err, pgx.ErrNoRows) {
return database.MediaItems{}, MethodNone, fmt.Errorf("resolve by sha256 (format): %w", err)
}
return database.MediaItems{}, MethodNone, pgx.ErrNoRows
}
+22 -2
View File
@@ -16,6 +16,10 @@ import (
"github.com/jackc/pgx/v5/pgtype"
)
// defaultConversionCacheTTL is the fallback kepub cache lifetime when no
// settings registry is wired. Matches the historical hardcoded 24h.
const defaultConversionCacheTTL = 24 * time.Hour
type ConvertedKEPUB struct {
Path string
SHA256 string
@@ -27,6 +31,7 @@ type ConversionService struct {
cacheDir string
conversionTool string
conversionCacheTTL time.Duration
settings *database.SettingsRegistry
}
func NewConversionService(db *database.Queries, cacheDir string) *ConversionService {
@@ -34,17 +39,32 @@ func NewConversionService(db *database.Queries, cacheDir string) *ConversionServ
db: db,
cacheDir: cacheDir,
conversionTool: "/usr/bin/kepubify",
conversionCacheTTL: 24 * time.Hour,
conversionCacheTTL: defaultConversionCacheTTL,
}
}
// SetSettings wires the tunable settings registry. When wired, the cache TTL
// is read live on each conversion request.
func (s *ConversionService) SetSettings(reg *database.SettingsRegistry) { s.settings = reg }
// cacheTTL returns the active conversion cache TTL.
func (s *ConversionService) cacheTTL() time.Duration {
if s.settings != nil {
return s.settings.ConversionCacheTTL()
}
if s.conversionCacheTTL > 0 {
return s.conversionCacheTTL
}
return defaultConversionCacheTTL
}
func (s *ConversionService) ConvertEPUBToKEPUB(ctx context.Context, mediaItemID pgtype.UUID, epubPath string) (*ConvertedKEPUB, error) {
existing, err := s.db.GetMediaItemFormatByType(ctx, database.GetMediaItemFormatByTypeParams{
MediaItemID: mediaItemID,
FormatType: "kepub",
})
if err == nil && existing.FilePath.Valid {
if time.Since(existing.CreatedAt.Time) < s.conversionCacheTTL {
if time.Since(existing.CreatedAt.Time) < s.cacheTTL() {
return &ConvertedKEPUB{
Path: existing.FilePath.String,
SHA256: existing.FileSha256.String,
+3 -1
View File
@@ -65,5 +65,7 @@ func TestConversionServiceDefaults(t *testing.T) {
assert.NotNil(t, service)
assert.Equal(t, cacheDir, service.cacheDir)
assert.Equal(t, "/usr/bin/kepubify", service.conversionTool)
assert.Equal(t, int64(24*3600*1000000000), service.conversionCacheTTL.Nanoseconds(), "Default TTL should be 24 hours")
assert.Equal(t, int64(24*3600*1000000000), service.conversionCacheTTL.Nanoseconds(), "Default TTL field should be 24 hours")
// cacheTTL() must reflect the same default when no registry is wired.
assert.Equal(t, int64(24*3600*1000000000), service.cacheTTL().Nanoseconds(), "Default TTL accessor should return 24 hours")
}
+119
View File
@@ -0,0 +1,119 @@
package services
import (
"bookhoard/internal/database"
"context"
"log"
"time"
"github.com/jackc/pgx/v5/pgtype"
)
// HashBackfillService is a one-time self-heal pass that computes and stores the
// SHA-256 for media items imported before hashing existed (file_sha256 IS
// NULL). It runs once shortly after startup, independently of auto-scan, and
// also performs a final conflict sweep that flags any content-duplicate groups
// (same library + SHA-256 at different paths) on the admin Hash Conflicts page.
//
// The sweep runs after the per-item pass because during the pass only one side
// of a preexisting duplicate pair may be hashed at a time - the group only
// becomes visible once every item has its hash.
type HashBackfillService struct {
db *database.Queries
libSvc *LibraryService
}
// NewHashBackfillService creates a backfill service.
func NewHashBackfillService(db *database.Queries) *HashBackfillService {
return &HashBackfillService{db: db, libSvc: NewLibraryService(db)}
}
// Run performs the backfill pass followed by the conflict sweep. It logs
// progress and never returns an error - failures on individual items are
// skipped so one unreadable file cannot block the rest.
func (s *HashBackfillService) Run(ctx context.Context) {
items, err := s.db.ListMediaItemsMissingHash(ctx)
if err != nil {
log.Printf("[HASH-BACKFILL] failed to list items missing hash: %v", err)
return
}
if len(items) == 0 {
log.Printf("[HASH-BACKFILL] all media items already hashed, nothing to do")
s.sweepConflicts(ctx)
return
}
log.Printf("[HASH-BACKFILL] computing SHA-256 for %d unhashed media items", len(items))
started := time.Now()
hashed, failed := 0, 0
for _, item := range items {
if ctx.Err() != nil {
log.Printf("[HASH-BACKFILL] cancelled after %d items", hashed)
return
}
path, err := s.libSvc.ResolveMediaPath(ctx, item.LibraryID, item.FilePath)
if err != nil {
log.Printf("[HASH-BACKFILL] could not resolve path for %q: %v", item.FilePath, err)
failed++
continue
}
sha, err := computeFileSHA256(path)
if err != nil {
log.Printf("[HASH-BACKFILL] could not hash %q: %v", path, err)
failed++
continue
}
_, err = s.db.UpdateMediaItemIdentifiers(ctx, database.UpdateMediaItemIdentifiersParams{
ID: item.ID,
FileSha256: pgtype.Text{String: sha, Valid: true},
HashConfidence: pgtype.Text{String: "sha256_full", Valid: true},
})
if err != nil {
log.Printf("[HASH-BACKFILL] could not store hash for %q: %v", item.FilePath, err)
failed++
continue
}
hashed++
if hashed%25 == 0 {
log.Printf("[HASH-BACKFILL] progress: %d/%d hashed", hashed, len(items))
}
}
log.Printf("[HASH-BACKFILL] done in %s: %d hashed, %d failed (of %d)",
time.Since(started).Round(time.Second), hashed, failed, len(items))
s.sweepConflicts(ctx)
}
// sweepConflicts flags every content-duplicate group (same library + SHA-256,
// more than one item) as a pending hash conflict. The upsert is a no-op for
// groups that are already tracked or resolved, so admins who chose "keep both"
// are never re-prompted.
func (s *HashBackfillService) sweepConflicts(ctx context.Context) {
groups, err := s.db.FindHashConflictGroups(ctx)
if err != nil {
log.Printf("[HASH-BACKFILL] conflict sweep failed: %v", err)
return
}
if len(groups) == 0 {
return
}
flagged := 0
for _, g := range groups {
if err := s.db.CreateHashConflict(ctx, database.CreateHashConflictParams{
LibraryID: g.LibraryID,
FileSha256: g.FileSha256.String,
}); err != nil {
log.Printf("[HASH-BACKFILL] could not record conflict group: %v", err)
continue
}
flagged++
}
log.Printf("[HASH-BACKFILL] flagged %d content-duplicate group(s) for admin review", flagged)
}
+165 -41
View File
@@ -146,16 +146,18 @@ type CalibreOPFMetadata struct {
Timestamp *time.Time
}
// NewMediaScanner creates a new media scanner instance
// NewMediaScanner creates a new media scanner instance.
//
// The fsnotify watcher is NOT created here. It is created lazily inside
// SetFolders only when watch=true (the long-lived watch-mode scanner).
// Ephemeral one-off scan jobs pass watch=false, so they never allocate a
// watcher (and thus can never panic on EMFILE/ENOSPC). This fixes the
// fd/inotify-watch leak where every scan job created a watcher that was
// never closed.
func NewMediaScanner(db *database.Queries) *MediaScanner {
watcher, err := fsnotify.NewWatcher()
if err != nil {
panic(fmt.Sprintf("Failed to create file watcher: %v", err))
}
return &MediaScanner{
db: db,
watcher: watcher,
watcher: nil,
settingsCache: NewSettingsCache(30 * time.Second),
dirtyDirs: make(map[string]time.Time),
fileStability: make(map[string]*atomic.Bool),
@@ -238,24 +240,34 @@ func (s *MediaScanner) GetStats() (int, int, int) {
return s.totalFiles, s.newItems, s.errors
}
func (s *MediaScanner) SetFolders(folders []string) error {
// SetFolders configures the scanner's folders and (optionally) sets up an
// fsnotify watcher over the full directory tree.
//
// watch should be true only for the single long-lived watch-mode scanner that
// actually consumes watcher.Events. Ephemeral scan jobs must pass false so no
// watcher (and thus no fd/inotify watches) is allocated — the watcher is never
// read by scan jobs and previously leaked one watcher per job.
func (s *MediaScanner) SetFolders(folders []string, watch bool) error {
s.folders = folders
// Remove old watch if exists
// Always close any previously-owned watcher so reconfiguration doesn't leak.
if s.watcher != nil {
if s.watcher != nil {
if err := s.watcher.Close(); err != nil {
fmt.Printf("Warning: failed to close old watcher during folder reconfiguration: %v\n", err)
}
if err := s.watcher.Close(); err != nil {
fmt.Printf("Warning: failed to close old watcher during folder reconfiguration: %v\n", err)
}
s.watcher = nil
}
// Create new watcher
watcher, err := fsnotify.NewWatcher()
if err != nil {
return fmt.Errorf("failed to create watcher: %v", err)
// Create + populate a fresh watcher only when the caller intends to read events.
if watch {
watcher, err := fsnotify.NewWatcher()
if err != nil {
// Return an error instead of panicking so a failed watcher can't
// take down the whole process.
return fmt.Errorf("failed to create watcher: %w", err)
}
s.watcher = watcher
}
s.watcher = watcher
// Build cache of allowed extensions per folder
// Uses Go AllowedExtensions map as source of truth (not DB)
@@ -286,31 +298,36 @@ func (s *MediaScanner) SetFolders(folders []string) error {
}
}
// Add all folders and their subdirectories to the watcher (like Audiobookshelf)
watchCount := 0
for _, folder := range folders {
if err := s.watcher.Add(folder); err != nil {
fmt.Printf("[WATCHER] Warning: failed to watch root folder %s: %v\n", folder, err)
} else {
watchCount++
}
filepath.WalkDir(folder, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if !d.IsDir() || path == folder {
return nil
}
if err := s.watcher.Add(path); err != nil {
fmt.Printf("[WATCHER] Warning: failed to watch subdirectory %s: %v\n", path, err)
// Add all folders and their subdirectories to the watcher (like Audiobookshelf).
// Only when watching; scan jobs (watch=false) skip this entirely.
if s.watcher != nil {
watchCount := 0
for _, folder := range folders {
if err := s.watcher.Add(folder); err != nil {
fmt.Printf("[WATCHER] Warning: failed to watch root folder %s: %v\n", folder, err)
} else {
watchCount++
}
return nil
})
}
filepath.WalkDir(folder, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if !d.IsDir() || path == folder {
return nil
}
if err := s.watcher.Add(path); err != nil {
fmt.Printf("[WATCHER] Warning: failed to watch subdirectory %s: %v\n", path, err)
} else {
watchCount++
}
return nil
})
}
fmt.Printf("[WATCHER] Now watching %d directories across %d root folders\n", watchCount, len(folders))
fmt.Printf("[WATCHER] Now watching %d directories across %d root folders\n", watchCount, len(folders))
} else {
fmt.Printf("[SCANNER] Configured %d root folders (watch mode disabled, no inotify watcher)\n", len(folders))
}
return nil
}
@@ -417,8 +434,10 @@ func (s *MediaScanner) ScanFolders(ctx context.Context) error {
}
if d.IsDir() {
if err := s.watcher.Add(path); err != nil {
fmt.Printf("Warning: failed to watch subdirectory %s: %v\n", path, err)
if s.watcher != nil {
if err := s.watcher.Add(path); err != nil {
fmt.Printf("Warning: failed to watch subdirectory %s: %v\n", path, err)
}
}
return nil
}
@@ -680,14 +699,24 @@ func (s *MediaScanner) processMediaFile(ctx context.Context, path string) (bool,
if err := s.updateMediaItem(ctx, existingItem.ID, path, info); err != nil {
fmt.Printf("Warning: failed to update existing media item: %v\n", err)
}
// Recompute hash identifiers too - a force rescan is the admin's
// backfill tool and must refresh stale or missing hashes.
s.recomputeHashInfo(ctx, existingItem.ID, libraryID, path)
return false, nil
} else {
// Normal behavior: check if file has changed (by size)
if existingItem.FileSize.Int64 != info.Size() {
fmt.Printf("File size changed, updating media item: %s\n", path)
_ = s.updateMediaItem(ctx, existingItem.ID, path, info)
// The bytes changed, so any stored hash is stale.
s.recomputeHashInfo(ctx, existingItem.ID, libraryID, path)
return false, nil
}
// Self-heal items imported before hashing existed: even an unchanged
// file gets its hash computed if missing.
if !existingItem.FileSha256.Valid || existingItem.FileSha256.String == "" {
s.recomputeHashInfo(ctx, existingItem.ID, libraryID, path)
}
fmt.Printf("Media item already exists with same size, skipping: %s\n", path)
return false, nil
}
@@ -716,6 +745,26 @@ func (s *MediaScanner) processMediaFile(ctx context.Context, path string) (bool,
path, hashInfo.FileSHA256, hashInfo.OPFIdentifier, hashInfo.OPFUUID, hashInfo.HashConfidence)
}
// Content dedup: if an item with the same SHA-256 already exists in this
// library (same file at a different path), treat it as existing rather than
// creating a duplicate. The file bytes are identical, so metadata matches.
if hashInfo.FileSHA256 != "" {
existingByHash, err := s.db.GetMediaItemBySHA256AndLibrary(ctx, database.GetMediaItemBySHA256AndLibraryParams{
FileSha256: pgtype.Text{String: hashInfo.FileSHA256, Valid: true},
LibraryID: libraryID,
})
if err == nil && existingByHash.ID.Valid {
fmt.Printf("Media item with same SHA-256 already exists in library (path %q), skipping duplicate: %s\n",
existingByHash.FilePath, path)
if s.forceRescan {
_ = s.updateMediaItem(ctx, existingByHash.ID, path, info)
}
return false, nil
} else if err != nil && !errors.Is(err, pgx.ErrNoRows) {
fmt.Printf("Warning: failed to check media item by SHA-256 for %s: %v\n", path, err)
}
}
// REMOVED: Comic metadata extraction now handled by mergeMetadata()
// This avoids duplicate extraction and ensures smart merging happens
@@ -2576,6 +2625,68 @@ func (s *MediaScanner) getMediaItemByFilePath(ctx context.Context, filePath stri
})
}
// recomputeHashInfo recomputes the file's hash identifiers and stores them on
// the media item (plus its per-format row). Called on force rescan, on file
// size change, and when an unchanged item is found with no stored hash, so
// items imported before hashing existed are backfilled by ordinary scans.
// After storing, it records a hash conflict if the same content now exists at
// more than one path in the library.
func (s *MediaScanner) recomputeHashInfo(ctx context.Context, mediaItemID pgtype.UUID, libraryID pgtype.UUID, path string) {
hashInfo, formatInfo, err := s.extractHashInfo(path)
if err != nil {
fmt.Printf("Warning: failed to extract hash info from %s: %v\n", path, err)
return
}
if hashInfo == nil || hashInfo.FileSHA256 == "" {
return
}
_, err = s.db.UpdateMediaItemIdentifiers(ctx, database.UpdateMediaItemIdentifiersParams{
ID: mediaItemID,
FileSha256: pgtype.Text{String: hashInfo.FileSHA256, Valid: true},
OpfIdentifier: pgtype.Text{String: hashInfo.OPFIdentifier, Valid: hashInfo.OPFIdentifier != ""},
OpfUuid: pgtype.Text{String: hashInfo.OPFUUID, Valid: hashInfo.OPFUUID != ""},
HashConfidence: pgtype.Text{String: hashInfo.HashConfidence, Valid: hashInfo.HashConfidence != ""},
})
if err != nil {
fmt.Printf("Warning: failed to update hash identifiers for %s: %v\n", path, err)
return
}
if formatInfo != nil {
_, _ = s.db.CreateMediaItemFormat(ctx, database.CreateMediaItemFormatParams{
MediaItemID: mediaItemID,
FormatType: formatInfo.FormatType,
FilePath: pgtype.Text{String: s.getRelativePath(formatInfo.FilePath), Valid: true},
FileSha256: pgtype.Text{String: formatInfo.FileSHA256, Valid: true},
FileSizeBytes: pgtype.Int8{Int64: formatInfo.FileSizeBytes, Valid: true},
MimeType: pgtype.Text{String: formatInfo.MimeType, Valid: true},
})
}
s.recordHashConflictIfAny(ctx, libraryID, hashInfo.FileSHA256)
}
// recordHashConflictIfAny flags a pending hash conflict when the given content
// hash is now shared by more than one media item in the same library. The
// upsert is a no-op for already-tracked (including resolved) groups.
func (s *MediaScanner) recordHashConflictIfAny(ctx context.Context, libraryID pgtype.UUID, fileSHA256 string) {
items, err := s.db.ListMediaItemsBySHA256AndLibrary(ctx, database.ListMediaItemsBySHA256AndLibraryParams{
FileSha256: pgtype.Text{String: fileSHA256, Valid: true},
LibraryID: libraryID,
})
if err != nil {
return
}
if len(items) > 1 {
fmt.Printf("Hash conflict: %d media items share SHA-256 %s in one library\n", len(items), fileSHA256)
_ = s.db.CreateHashConflict(ctx, database.CreateHashConflictParams{
LibraryID: libraryID,
FileSha256: fileSHA256,
})
}
}
func (s *MediaScanner) getMimeType(path string) string {
ext := strings.ToLower(filepath.Ext(path))
if mime, ok := MimeTypes[ext]; ok {
@@ -2601,6 +2712,13 @@ func (s *MediaScanner) WatchChanges(ctx context.Context) error {
go s.startBackupScan(ctx)
go func() {
// The event loop only runs if a real watcher was set up (watch=true).
// If watching with no watcher (e.g. inotify unavailable through a Docker
// bind mount), polling via startBackupScan above still handles detection.
if s.watcher == nil {
fmt.Printf("[WATCHER] No inotify watcher configured; relying on periodic polling for change detection\n")
return
}
fmt.Printf("[WATCHER] Event loop started for %d folders\n", len(s.folders))
for {
select {
@@ -2992,6 +3110,12 @@ func (s *MediaScanner) startBackupScan(ctx context.Context) {
}
func (s *MediaScanner) calculateFileSHA256(filePath string) (string, error) {
return computeFileSHA256(filePath)
}
// computeFileSHA256 is the package-level full-file SHA-256 used by the hash
// backfill service; the MediaScanner method delegates to it.
func computeFileSHA256(filePath string) (string, error) {
file, err := os.Open(filePath)
if err != nil {
return "", fmt.Errorf("failed to open file: %v", err)
+17 -11
View File
@@ -392,17 +392,23 @@ func (s *ReaderService) UpdateSettings(
func (s *ReaderService) getDefaultSettings() map[string]interface{} {
return map[string]interface{}{
"chrome_behavior": "auto-hide",
"progress_mode": "pages",
"chrome_theme": "tokyo-night",
"reading_theme": "dark",
"reading_font": "literata",
"font_size": 16,
"line_height": 1.6,
"margin_width": 20,
"tap_zone_size": 30,
"auto_scroll": false,
"panel_zoom_enabled": true,
"chrome_behavior": "auto-hide",
"progress_mode": "pages",
"chrome_theme": "tokyo-night",
"reading_theme": "dark",
"reading_font": "literata",
"font_size": 16,
"line_height": 1.6,
"margin_width": 20,
"tap_zone_size": 30,
"auto_scroll": false,
"panel_zoom_enabled": true,
"double_page_spread": true,
"pdf_interaction_mode": "select",
"fx_brightness": 1,
"fx_contrast": 1,
"fx_invert": false,
"tap_zones_enabled": true,
// Dockable panel defaults
"panel_layout": map[string]interface{}{
+32 -5
View File
@@ -176,10 +176,17 @@ func (w *Worker) GetActiveJobCount() int {
return count
}
func NewWorker(numWorkers int, connManager *wsync.ConnectionManager) *Worker {
return NewWorkerWithConfig(numWorkers, 100, connManager)
}
// NewWorkerWithConfig constructs a worker pool with the given worker count and
// job-queue capacity. Used at startup to source values from the settings
// registry.
func NewWorkerWithConfig(numWorkers, queueCap int, connManager *wsync.ConnectionManager) *Worker {
ctx, cancel := context.WithCancel(context.Background())
w := &Worker{
jobQueue: make(chan *Job, 100),
jobQueue: make(chan *Job, queueCap),
results: make(map[string]*JobResult),
ctx: ctx,
cancel: cancel,
@@ -205,7 +212,23 @@ func (w *Worker) worker() {
return
}
w.processJob(job)
// Recover from any panic inside a job so a single failing job can
// never crash the whole worker goroutine (and thus the process).
func() {
defer func() {
if r := recover(); r != nil {
fmt.Printf("[WORKER] panic in job %s (%s): %v\n", job.ID, job.Type, r)
w.mu.Lock()
w.results[job.ID] = &JobResult{
JobID: job.ID,
Status: JobStatusFailed,
Error: fmt.Sprintf("panic: %v", r),
}
w.mu.Unlock()
}
}()
w.processJob(job)
}()
case <-w.ctx.Done():
return
@@ -348,6 +371,7 @@ func (w *Worker) processScanJob(job *Job) (interface{}, error) {
}
scanner := NewMediaScanner(db)
defer scanner.Close()
scanner.job = job
job.ProgressCallback = func(progress float64, filesScanned, newItems, errors int) {
@@ -377,7 +401,7 @@ func (w *Worker) processScanJob(job *Job) (interface{}, error) {
}
}
if err := scanner.SetFolders(folders); err != nil {
if err := scanner.SetFolders(folders, false); err != nil {
return nil, err
}
@@ -521,7 +545,8 @@ func (w *Worker) processSetFoldersJob(job *Job) (interface{}, error) {
// Create scanner and configure folders
scanner := NewMediaScanner(db)
if err := scanner.SetFolders(folders); err != nil {
defer scanner.Close()
if err := scanner.SetFolders(folders, false); err != nil {
return nil, fmt.Errorf("failed to set folders: %w", err)
}
@@ -900,6 +925,7 @@ func (w *Worker) processDirectoryScanJob(job *Job) (interface{}, error) {
// Create temporary scanner instance for this job
scanner := NewMediaScanner(db)
defer scanner.Close()
scanner.job = job
// Find which library owns this directory (prefix match for subdirectories)
ctx := context.Background()
@@ -918,7 +944,7 @@ func (w *Worker) processDirectoryScanJob(job *Job) (interface{}, error) {
folderPaths = append(folderPaths, f.FolderPath)
}
// Configure scanner with folders
if err := scanner.SetFolders(folderPaths); err != nil {
if err := scanner.SetFolders(folderPaths, false); err != nil {
return nil, fmt.Errorf("failed to set folders: %w", err)
}
// Now scan the directory
@@ -976,3 +1002,4 @@ func (w *Worker) Shutdown() {
close(w.jobQueue)
w.wg.Wait()
}
+27 -10
View File
@@ -1,8 +1,9 @@
// Package setupstatus reports whether the application's initial setup has been
// completed. Setup is considered complete as soon as at least one admin user
// exists, regardless of how that user was created (setup wizard, API, or a
// future CLI). This keeps the setup gate a derived property of real data
// rather than a manually-flipped flag that can drift out of sync.
// completed. Setup is considered complete when at least one admin user exists
// AND a non-empty base_url has been configured, regardless of how those were
// created (setup wizard, API, or a future CLI). This keeps the setup gate a
// derived property of real data rather than a manually-flipped flag that can
// drift out of sync.
package setupstatus
import (
@@ -18,6 +19,12 @@ type AdminCounter interface {
CountAdmins(ctx context.Context) (int64, error)
}
// BaseURLGetter returns the configured base_url value from the database, or an
// error if it cannot be read. Defined as a function type (not an interface) so
// it can be satisfied by a closure wrapping *database.Queries.GetSystemConfig
// without importing the database package.
type BaseURLGetter func(ctx context.Context) (string, error)
var (
cacheMu sync.RWMutex
cacheComplete bool = true
@@ -26,10 +33,11 @@ var (
)
// IsSetupComplete reports whether setup is complete. Setup is complete when at
// least one admin user exists. A short in-memory cache avoids hammering the
// database on every request. On a database error the function fails open
// (returns true) so a transient outage does not lock users out of the app.
func IsSetupComplete(ctx context.Context, q AdminCounter) bool {
// least one admin user exists AND base_url is configured. A short in-memory
// cache avoids hammering the database on every request. On a database error the
// function fails open (returns true) so a transient outage does not lock users
// out of the app.
func IsSetupComplete(ctx context.Context, q AdminCounter, baseURLGetter BaseURLGetter) bool {
cacheMu.RLock()
if time.Now().Before(cacheExpiry) {
complete := cacheComplete
@@ -38,12 +46,20 @@ func IsSetupComplete(ctx context.Context, q AdminCounter) bool {
}
cacheMu.RUnlock()
count, err := q.CountAdmins(ctx)
complete := true
count, err := q.CountAdmins(ctx)
if err == nil {
complete = count > 0
}
if complete && baseURLGetter != nil {
baseURL, err := baseURLGetter(ctx)
if err == nil {
complete = baseURL != ""
}
}
cacheMu.Lock()
cacheComplete = complete
cacheExpiry = time.Now().Add(cacheTTL)
@@ -53,7 +69,8 @@ func IsSetupComplete(ctx context.Context, q AdminCounter) bool {
// Invalidate clears the cached setup status so the next call to IsSetupComplete
// re-reads from the database. Call this after any write that could change the
// admin user count (user creation, role promotion/demotion, user deletion).
// admin user count (user creation, role promotion/demotion, user deletion) or
// the base_url configuration.
func Invalidate() {
cacheMu.Lock()
cacheComplete = true
+258 -74
View File
@@ -18,8 +18,38 @@ import (
"github.com/jackc/pgx/v5/pgtype"
)
// TombstoneTTL is the fallback retention for soft-deleted annotations when no
// settings registry is wired (e.g. in tests). It matches the historical value.
const TombstoneTTL = 30 * 24 * time.Hour
type AnnotationService struct {
db *database.Queries
connMgr *ConnectionManager
settings *database.SettingsRegistry
}
func NewAnnotationService(db *database.Queries, connMgr *ConnectionManager) *AnnotationService {
return &AnnotationService{db: db, connMgr: connMgr}
}
// SetSettings wires the tunable settings registry. When wired, the tombstone
// TTL is read live from the DB; otherwise the package const TombstoneTTL is
// used.
func (s *AnnotationService) SetSettings(reg *database.SettingsRegistry) { s.settings = reg }
// tombstoneTTL returns the active tombstone retention window.
func (s *AnnotationService) tombstoneTTL() time.Duration {
if s.settings != nil {
return s.settings.TombstoneTTL()
}
return TombstoneTTL
}
// ActiveTombstoneTTL exposes the configured tombstone retention window for
// callers outside the sync package (e.g. kobo/koreader handlers) that need to
// compute cutoffs consistently with the service.
func (s *AnnotationService) ActiveTombstoneTTL() time.Duration { return s.tombstoneTTL() }
type SaveOutcome string
const (
@@ -29,15 +59,6 @@ const (
SaveOutcomeDeleted SaveOutcome = "deleted"
)
type AnnotationService struct {
db *database.Queries
connMgr *ConnectionManager
}
func NewAnnotationService(db *database.Queries, connMgr *ConnectionManager) *AnnotationService {
return &AnnotationService{db: db, connMgr: connMgr}
}
type SaveHighlightRequest struct {
MediaItemID pgtype.UUID
UserID pgtype.UUID
@@ -54,6 +75,11 @@ type SaveHighlightRequest struct {
Source string
ModifiedAt time.Time
DeviceSyncData json.RawMessage
// DedupKey overrides the computed key when the client echoes back an
// annotation it received from us (device echoes carry device-native
// locators, so the computed key would never match the original row and
// every pull→push cycle would mint a duplicate).
DedupKey string
}
type SaveHighlightResult struct {
@@ -63,7 +89,10 @@ type SaveHighlightResult struct {
}
func (s *AnnotationService) SaveHighlight(ctx context.Context, req SaveHighlightRequest) (*SaveHighlightResult, error) {
dedupKey := ComputeDedupKey(req.SelectionText, req.EpubcfiStart, req.StartPosition)
dedupKey := req.DedupKey
if dedupKey == "" {
dedupKey = ComputeDedupKey(req.SelectionText, req.EpubcfiStart, req.StartPosition)
}
existing, err := s.db.GetMediaHighlightByDedupKey(ctx, database.GetMediaHighlightByDedupKeyParams{
UserID: req.UserID,
@@ -79,10 +108,12 @@ func (s *AnnotationService) SaveHighlight(ctx context.Context, req SaveHighlight
}
if existing.Deleted.Bool {
if existing.DeletedAt.Valid && time.Since(existing.DeletedAt.Time) < TombstoneTTL {
if !incomingNewerThanTombstone(req.ModifiedAt, existing.DeletedAt, existing.LastModifiedAt) {
return &SaveHighlightResult{Highlight: existing, Outcome: SaveOutcomeDeleted}, nil
}
return s.createHighlight(ctx, req, dedupKey)
// Newer than the tombstone: a deliberate re-create. Resurrect via the
// LWW update (which clears deleted/deleted_at).
return s.applyLWW(ctx, req, existing, dedupKey)
}
return s.applyLWW(ctx, req, existing, dedupKey)
@@ -101,22 +132,22 @@ func (s *AnnotationService) createHighlight(
deviceData := mergeDeviceSyncData(nil, req.Source, req.DeviceSyncData)
highlight, err := s.db.CreateMediaHighlightFull(ctx, database.CreateMediaHighlightFullParams{
MediaItemID: req.MediaItemID,
UserID: req.UserID,
SelectionText: req.SelectionText,
StartPosition: pgText(req.StartPosition),
EndPosition: pgText(req.EndPosition),
Color: pgText(req.Color),
NoteText: pgText(req.NoteText),
PercentageStart: pgFloat8(req.PercentageStart),
PercentageEnd: pgFloat8(req.PercentageEnd),
EpubcfiStart: pgText(req.EpubcfiStart),
EpubcfiEnd: pgText(req.EpubcfiEnd),
ChapterReference: pgInt4(req.ChapterReference),
DedupKey: pgtype.Text{String: dedupKey, Valid: true},
LastModifiedAt: pgtype.Timestamptz{Time: modifiedAt, Valid: true},
MediaItemID: req.MediaItemID,
UserID: req.UserID,
SelectionText: req.SelectionText,
StartPosition: pgText(req.StartPosition),
EndPosition: pgText(req.EndPosition),
Color: pgText(req.Color),
NoteText: pgText(req.NoteText),
PercentageStart: pgFloat8(req.PercentageStart),
PercentageEnd: pgFloat8(req.PercentageEnd),
EpubcfiStart: pgText(req.EpubcfiStart),
EpubcfiEnd: pgText(req.EpubcfiEnd),
ChapterReference: pgInt4(req.ChapterReference),
DedupKey: pgtype.Text{String: dedupKey, Valid: true},
LastModifiedAt: pgtype.Timestamptz{Time: modifiedAt, Valid: true},
LastModifiedSource: pgtype.Text{String: req.Source, Valid: req.Source != ""},
DeviceSyncData: deviceData,
DeviceSyncData: deviceData,
})
if err != nil {
return nil, fmt.Errorf("create highlight: %w", err)
@@ -156,20 +187,20 @@ func (s *AnnotationService) applyLWW(
deviceData := mergeDeviceSyncData(existing.DeviceSyncData, req.Source, req.DeviceSyncData)
highlight, err := s.db.UpdateMediaHighlightForSync(ctx, database.UpdateMediaHighlightForSyncParams{
ID: existing.ID,
SelectionText: req.SelectionText,
StartPosition: pgText(req.StartPosition),
EndPosition: pgText(req.EndPosition),
Color: pgText(req.Color),
NoteText: pgText(req.NoteText),
PercentageStart: pgFloat8(req.PercentageStart),
PercentageEnd: pgFloat8(req.PercentageEnd),
EpubcfiStart: pgText(req.EpubcfiStart),
EpubcfiEnd: pgText(req.EpubcfiEnd),
ChapterReference: pgInt4(req.ChapterReference),
LastModifiedAt: pgtype.Timestamptz{Time: modifiedAt, Valid: true},
ID: existing.ID,
SelectionText: req.SelectionText,
StartPosition: pgText(req.StartPosition),
EndPosition: pgText(req.EndPosition),
Color: pgText(req.Color),
NoteText: pgText(req.NoteText),
PercentageStart: pgFloat8(req.PercentageStart),
PercentageEnd: pgFloat8(req.PercentageEnd),
EpubcfiStart: pgText(req.EpubcfiStart),
EpubcfiEnd: pgText(req.EpubcfiEnd),
ChapterReference: pgInt4(req.ChapterReference),
LastModifiedAt: pgtype.Timestamptz{Time: modifiedAt, Valid: true},
LastModifiedSource: pgtype.Text{String: req.Source, Valid: req.Source != ""},
DeviceSyncData: deviceData,
DeviceSyncData: deviceData,
})
if err != nil {
return nil, fmt.Errorf("update highlight: %w", err)
@@ -237,8 +268,101 @@ func (s *AnnotationService) TombstoneHighlightByID(
return nil
}
// TombstoneBookmarkByDedupKey soft-deletes a bookmark by its dedup key — the
// device-sync counterpart of TombstoneHighlight. Devices report deletions by
// dedup key (they have no row IDs), so this keeps bookmark delete propagation
// symmetric with highlights.
func (s *AnnotationService) TombstoneBookmarkByDedupKey(
ctx context.Context,
userID, mediaItemID pgtype.UUID,
dedupKey string,
source string,
) error {
if dedupKey == "" {
return nil
}
err := s.db.TombstoneMediaBookmarkByDedupKey(ctx, database.TombstoneMediaBookmarkByDedupKeyParams{
UserID: userID,
MediaItemID: mediaItemID,
DedupKey: pgtype.Text{String: dedupKey, Valid: true},
})
if err != nil {
return fmt.Errorf("tombstone bookmark: %w", err)
}
s.broadcast(pgtype.UUID{}, userID, mediaItemID, "bookmark_delete", source)
return nil
}
// ValidAnnotationKind reports whether kind is one of the annotation types
// accepted by the history restore/purge endpoints.
func ValidAnnotationKind(kind string) bool {
return kind == "highlight" || kind == "note" || kind == "bookmark"
}
// RestoreAnnotationByID clears the tombstone on a deleted annotation,
// returning it to the active set. The row itself was never removed, so
// restoration is lossless. Returns false when no matching deleted annotation
// exists (wrong owner, wrong book, or not actually deleted).
func (s *AnnotationService) RestoreAnnotationByID(
ctx context.Context,
kind string,
userID, mediaItemID, annotationID pgtype.UUID,
) (bool, error) {
var rows int64
var err error
switch kind {
case "highlight":
rows, err = s.db.RestoreMediaHighlightByID(ctx, database.RestoreMediaHighlightByIDParams{
ID: annotationID, UserID: userID, MediaItemID: mediaItemID})
case "note":
rows, err = s.db.RestoreMediaNoteByID(ctx, database.RestoreMediaNoteByIDParams{
ID: annotationID, UserID: userID, MediaItemID: mediaItemID})
case "bookmark":
rows, err = s.db.RestoreMediaBookmarkByID(ctx, database.RestoreMediaBookmarkByIDParams{
ID: annotationID, UserID: userID, MediaItemID: mediaItemID})
default:
return false, fmt.Errorf("unknown annotation kind: %s", kind)
}
if err != nil {
return false, fmt.Errorf("restore %s: %w", kind, err)
}
if rows > 0 {
s.broadcast(annotationID, userID, mediaItemID, kind, "web")
}
return rows > 0, nil
}
// PurgeAnnotationByID permanently deletes an already-tombstoned annotation
// from the history. Unlike a tombstone this is irreversible; the TTL-driven
// maintenance sweep does the same thing to old tombstones eventually.
func (s *AnnotationService) PurgeAnnotationByID(
ctx context.Context,
kind string,
userID, mediaItemID, annotationID pgtype.UUID,
) (bool, error) {
var rows int64
var err error
switch kind {
case "highlight":
rows, err = s.db.PurgeMediaHighlightByID(ctx, database.PurgeMediaHighlightByIDParams{
ID: annotationID, UserID: userID, MediaItemID: mediaItemID})
case "note":
rows, err = s.db.PurgeMediaNoteByID(ctx, database.PurgeMediaNoteByIDParams{
ID: annotationID, UserID: userID, MediaItemID: mediaItemID})
case "bookmark":
rows, err = s.db.PurgeMediaBookmarkByID(ctx, database.PurgeMediaBookmarkByIDParams{
ID: annotationID, UserID: userID, MediaItemID: mediaItemID})
default:
return false, fmt.Errorf("unknown annotation kind: %s", kind)
}
if err != nil {
return false, fmt.Errorf("purge %s: %w", kind, err)
}
return rows > 0, nil
}
func (s *AnnotationService) PurgeExpiredTombstones(ctx context.Context) error {
cutoff := pgtype.Timestamptz{Time: time.Now().Add(-TombstoneTTL), Valid: true}
cutoff := pgtype.Timestamptz{Time: time.Now().Add(-s.tombstoneTTL()), Valid: true}
if err := s.db.PurgeExpiredHighlightTombstones(ctx, cutoff); err != nil {
return fmt.Errorf("purge highlight tombstones: %w", err)
}
@@ -251,7 +375,13 @@ func (s *AnnotationService) PurgeExpiredTombstones(ctx context.Context) error {
return nil
}
func (s *AnnotationService) StartTombstonePurger() context.CancelFunc {
// StartDailyMaintenance launches a single background goroutine that runs all
// periodic cleanup tasks once every 24 hours: expired annotation tombstones,
// expired/revoked refresh tokens (retention follows the configured session
// duration), and expired OPDS tokens. Each task is independent; a failure in
// one is logged and does not skip the others. The returned CancelFunc stops the
// goroutine and the underlying ticker; it must be invoked on shutdown.
func (s *AnnotationService) StartDailyMaintenance() context.CancelFunc {
ticker := time.NewTicker(24 * time.Hour)
ctx, cancel := context.WithCancel(context.Background())
@@ -262,9 +392,7 @@ func (s *AnnotationService) StartTombstonePurger() context.CancelFunc {
ticker.Stop()
return
case <-ticker.C:
if err := s.PurgeExpiredTombstones(ctx); err != nil {
log.Printf("AnnotationService: tombstone purge failed: %v", err)
}
s.runDailyMaintenance(ctx)
}
}
}()
@@ -272,6 +400,28 @@ func (s *AnnotationService) StartTombstonePurger() context.CancelFunc {
return cancel
}
// runDailyMaintenance executes every periodic cleanup task. Tasks run
// sequentially under the single daily-tick goroutine so there is no added
// concurrency. All three queries only delete rows that are already unusable
// (expired or revoked), so this never logs out active sessions.
func (s *AnnotationService) runDailyMaintenance(ctx context.Context) {
if err := s.PurgeExpiredTombstones(ctx); err != nil {
log.Printf("maintenance: tombstone purge failed: %v", err)
}
if err := s.db.CleanupExpiredOpdsTokens(ctx); err != nil {
log.Printf("maintenance: OPDS token purge failed: %v", err)
}
// Refresh-token retention follows the configured session duration; re-read
// on every tick so live settings changes are honored. Guarded so unwired
// test paths simply skip cleanup (production always wires the registry).
if s.settings != nil {
retention := s.settings.SessionDuration().Seconds()
if err := s.db.CleanupExpiredRefreshTokens(ctx, retention); err != nil {
log.Printf("maintenance: refresh token purge failed: %v", err)
}
}
}
type SaveNoteRequest struct {
MediaItemID pgtype.UUID
UserID pgtype.UUID
@@ -286,6 +436,7 @@ type SaveNoteRequest struct {
Source string
ModifiedAt time.Time
DeviceSyncData []byte
DedupKey string // overrides the computed key for device echoes
}
type SaveNoteResult struct {
@@ -299,7 +450,10 @@ func (s *AnnotationService) SaveNote(ctx context.Context, req SaveNoteRequest) (
return nil, errors.New("invalid user_id or media_item_id")
}
dedupKey := ComputeDedupKey(req.Content, req.EpubcfiLocation, req.Position)
dedupKey := req.DedupKey
if dedupKey == "" {
dedupKey = ComputeDedupKey(req.Content, req.EpubcfiLocation, req.Position)
}
existing, err := s.db.GetMediaNoteByDedupKey(ctx, database.GetMediaNoteByDedupKeyParams{
UserID: req.UserID,
@@ -314,7 +468,11 @@ func (s *AnnotationService) SaveNote(ctx context.Context, req SaveNoteRequest) (
}
if existing.Deleted.Valid && existing.Deleted.Bool {
return &SaveNoteResult{Note: existing, Outcome: SaveOutcomeDeleted}, nil
if !incomingNewerThanTombstone(req.ModifiedAt, existing.DeletedAt, existing.LastModifiedAt) {
return &SaveNoteResult{Note: existing, Outcome: SaveOutcomeDeleted}, nil
}
// Newer than the tombstone: a deliberate re-create. Resurrect.
return s.applyNoteLWW(ctx, req, existing, dedupKey)
}
return s.applyNoteLWW(ctx, req, existing, dedupKey)
@@ -432,6 +590,9 @@ type SaveBookmarkRequest struct {
Source string
ModifiedAt time.Time
DeviceSyncData json.RawMessage
// DedupKey overrides the computed key for device echoes (see
// SaveHighlightRequest).
DedupKey string
}
type SaveBookmarkResult struct {
@@ -441,7 +602,10 @@ type SaveBookmarkResult struct {
}
func (s *AnnotationService) SaveBookmark(ctx context.Context, req SaveBookmarkRequest) (*SaveBookmarkResult, error) {
dedupKey := ComputeDedupKey(req.Title, req.EpubcfiLocation, req.Position)
dedupKey := req.DedupKey
if dedupKey == "" {
dedupKey = ComputeDedupKey(req.Title, req.EpubcfiLocation, req.Position)
}
existing, err := s.db.GetMediaBookmarkByDedupKey(ctx, database.GetMediaBookmarkByDedupKeyParams{
UserID: req.UserID,
@@ -457,10 +621,13 @@ func (s *AnnotationService) SaveBookmark(ctx context.Context, req SaveBookmarkRe
}
if existing.Deleted.Bool {
if existing.DeletedAt.Valid && time.Since(existing.DeletedAt.Time) < TombstoneTTL {
if !incomingNewerThanTombstone(req.ModifiedAt, existing.DeletedAt, existing.LastModifiedAt) {
return &SaveBookmarkResult{Bookmark: existing, Outcome: SaveOutcomeDeleted}, nil
}
return s.createBookmark(ctx, req, dedupKey)
// Newer than the tombstone: a deliberate re-create. Resurrect via the
// LWW update instead of INSERT (the tombstoned row still holds the
// UNIQUE(media_item_id, user_id, title) slot).
return s.applyBookmarkLWW(ctx, req, existing, dedupKey)
}
return s.applyBookmarkLWW(ctx, req, existing, dedupKey)
@@ -474,21 +641,21 @@ func (s *AnnotationService) createBookmark(ctx context.Context, req SaveBookmark
deviceData := mergeDeviceSyncData(nil, req.Source, req.DeviceSyncData)
bm, err := s.db.CreateMediaBookmarkFull(ctx, database.CreateMediaBookmarkFullParams{
MediaItemID: req.MediaItemID,
UserID: req.UserID,
PageNumber: pgInt4(req.PageNumber),
ChapterNumber: pgInt4(req.ChapterNumber),
CfiPosition: pgText(req.CFIPosition),
Title: req.Title,
Position: pgText(req.Position),
Notes: pgText(req.Notes),
MediaItemID: req.MediaItemID,
UserID: req.UserID,
PageNumber: pgInt4(req.PageNumber),
ChapterNumber: pgInt4(req.ChapterNumber),
CfiPosition: pgText(req.CFIPosition),
Title: req.Title,
Position: pgText(req.Position),
Notes: pgText(req.Notes),
PercentageLocation: pgFloat8(req.PercentageLoc),
EpubcfiLocation: pgText(req.EpubcfiLocation),
ChapterReference: pgInt4(req.ChapterReference),
DedupKey: pgtype.Text{String: dedupKey, Valid: true},
LastModifiedAt: pgtype.Timestamptz{Time: modifiedAt, Valid: true},
EpubcfiLocation: pgText(req.EpubcfiLocation),
ChapterReference: pgInt4(req.ChapterReference),
DedupKey: pgtype.Text{String: dedupKey, Valid: true},
LastModifiedAt: pgtype.Timestamptz{Time: modifiedAt, Valid: true},
LastModifiedSource: pgtype.Text{String: req.Source, Valid: req.Source != ""},
DeviceSyncData: deviceData,
DeviceSyncData: deviceData,
})
if err != nil {
return nil, fmt.Errorf("create bookmark: %w", err)
@@ -517,19 +684,19 @@ func (s *AnnotationService) applyBookmarkLWW(ctx context.Context, req SaveBookma
deviceData := mergeDeviceSyncData(existing.DeviceSyncData, req.Source, req.DeviceSyncData)
bm, err := s.db.UpdateMediaBookmarkForSync(ctx, database.UpdateMediaBookmarkForSyncParams{
ID: existing.ID,
PageNumber: pgInt4(req.PageNumber),
ChapterNumber: pgInt4(req.ChapterNumber),
CfiPosition: pgText(req.CFIPosition),
Title: req.Title,
Position: pgText(req.Position),
Notes: pgText(req.Notes),
ID: existing.ID,
PageNumber: pgInt4(req.PageNumber),
ChapterNumber: pgInt4(req.ChapterNumber),
CfiPosition: pgText(req.CFIPosition),
Title: req.Title,
Position: pgText(req.Position),
Notes: pgText(req.Notes),
PercentageLocation: pgFloat8(req.PercentageLoc),
EpubcfiLocation: pgText(req.EpubcfiLocation),
ChapterReference: pgInt4(req.ChapterReference),
LastModifiedAt: pgtype.Timestamptz{Time: modifiedAt, Valid: true},
EpubcfiLocation: pgText(req.EpubcfiLocation),
ChapterReference: pgInt4(req.ChapterReference),
LastModifiedAt: pgtype.Timestamptz{Time: modifiedAt, Valid: true},
LastModifiedSource: pgtype.Text{String: req.Source, Valid: req.Source != ""},
DeviceSyncData: deviceData,
DeviceSyncData: deviceData,
})
if err != nil {
return nil, fmt.Errorf("update bookmark: %w", err)
@@ -670,6 +837,23 @@ func ComputeDedupKey(selectionText, epubcfiStart, startPosition string) string {
return hex.EncodeToString(h.Sum(nil))
}
// incomingNewerThanTombstone reports whether an incoming save should
// resurrect a tombstoned annotation. A save carrying a modification time
// newer than the tombstone (e.g. the user deliberately re-adding on the web,
// or a device that genuinely re-created it) wins; a save with a missing or
// older timestamp is treated as a stale replay from a client that still has
// the deleted annotation, and the tombstone stands.
func incomingNewerThanTombstone(incoming time.Time, deletedAt, lastModifiedAt pgtype.Timestamptz) bool {
if incoming.IsZero() {
return false
}
tombstone := deletedAt.Time
if lastModifiedAt.Valid && lastModifiedAt.Time.After(tombstone) {
tombstone = lastModifiedAt.Time
}
return incoming.After(tombstone)
}
func normalizeText(s string) string {
fields := strings.Fields(strings.ToLower(s))
return strings.Join(fields, " ")

Some files were not shown because too many files have changed in this diff Show More