From 12aab156f6ce4e5a1169cface098c845379b00d2 Mon Sep 17 00:00:00 2001 From: John O'Keefe Date: Mon, 2 Nov 2026 10:12:41 -0500 Subject: [PATCH] docs(user): sync-guide rebuild to today's platform; calibre guide de-API'd sync-guide.md was 591 lines with good implementation-derived bones (matching confidence table, universal locations, conflict flow) wrapped around a stale device story: Android 'coming later' (the app exists and self-registers), a deleted queue UI, invented UI (color status badges, Auto-Resolve Future, Priority Sync buttons), placeholder support links, an HTTPS-1.3 overclaim on LAN setups that are plain HTTP by design, and a changelog promising the future. Rebuilt to ~140 lines, every claim grounded: device table matches reality (web/Android/KOReader supported, Kobo native server-complete with rollout pending), resolution strategies as the API actually defines them (device / server / highest_progress), offline queuing honestly scoped (Android queues; server holds per-device queue; no dedicated queue UI today), reading history on Progress page, RLS-era privacy wording, device limit 10 default admin-settable. Reader Sync Progress button verified. calibre-integration.md: mostly current (sidecar-first model, Dublin Core tables, Write-to-file section all kept); fixed the user-hostile 'POST /api/scanner/scan' instructions to point at the admin scan controls, and dropped the yourusername placeholder repo links. --- docs/user/calibre-integration.md | 9 +- docs/user/sync-guide.md | 653 +++++-------------------------- 2 files changed, 102 insertions(+), 560 deletions(-) diff --git a/docs/user/calibre-integration.md b/docs/user/calibre-integration.md index 66c31832..9b9beeaa 100644 --- a/docs/user/calibre-integration.md +++ b/docs/user/calibre-integration.md @@ -139,7 +139,7 @@ As long as a `metadata.opf` file exists in the folder, Bookhoard will import the **Steps**: 1. Add the Calibre library folder in Bookhoard (Administration → Libraries → expand the library → **Folders**) -2. Trigger a scan via the **Scanner API**, or let watch mode pick up the changed files (the File Watcher status is shown on the admin dashboard) +2. Trigger a scan from the admin interface's scan controls, or let watch mode pick up the changed files automatically 3. Bookhoard imports all 500 books with: - Correct titles and authors - Series information (e.g., "Harry Potter #2") @@ -168,7 +168,7 @@ As long as a `metadata.opf` file exists in the folder, Bookhoard will import the **Steps**: 1. Edit metadata in Calibre (it updates `metadata.opf`) 2. In Bookhoard, trigger a rescan: - - Via the **Scanner API** (`POST /api/scanner/scan`), or + - Via the admin interface's scan controls, or - Let watch mode detect the changed files automatically (see File Watcher on the admin dashboard) 3. Bookhoard detects updated `metadata.opf` and refreshes metadata @@ -183,7 +183,7 @@ As long as a `metadata.opf` file exists in the folder, Bookhoard will import the **Solutions**: 1. **Check file structure**: Ensure `metadata.opf` is in the same folder as the book file 2. **Verify library type**: Ensure library type matches content (ebook vs. audiobook) -3. **Force rescan**: Trigger a scan via the Scanner API to re-import all metadata (watch mode also picks up changed files automatically) +3. **Force rescan**: Trigger a rescan from the admin interface to re-import all metadata (watch mode also picks up changed files automatically) 4. **Check logs**: Review Bookhoard logs for parsing errors ### Incorrect Metadata @@ -332,11 +332,10 @@ file is never touched if anything fails (temp file + atomic rename). - [Calibre Website](https://calibre-ebook.com/) - [Calibre User Manual](https://manual.calibre-ebook.com/) - [Calibre Forum](https://www.mobileread.com/forums/forumdisplay.php?f=166) -- [Bookhoard GitHub](https://github.com/yourusername/bookhoard) ## Need Help? - **Documentation**: See [User Guide](user-guide.md) for general Bookhoard usage - **Troubleshooting**: See [Operations Guide](../operations/operations.md) for common issues - **Developer Docs**: See [Developer Portal](../developer/development.md) for technical details -- **Issues**: Report bugs or request features on [GitHub Issues](https://github.com/yourusername/bookhoard/issues) +- **Issues**: Report bugs or request features on your repository's issue tracker diff --git a/docs/user/sync-guide.md b/docs/user/sync-guide.md index 40fa9a95..9b219f0b 100644 --- a/docs/user/sync-guide.md +++ b/docs/user/sync-guide.md @@ -1,591 +1,134 @@ -# Bookhoard Universal Sync - User Guide +# Universal Sync — how your devices stay in step -## Table of Contents +Sync is the promise that makes Bookhoard more than a web page: read on +whatever is in your hands, and every other device catches up. One server +holds the truth for each book — your position, bookmarks, highlights, and +notes — and every registered device pushes and pulls against it. -1. [What is Universal Sync?](#what-is-universal-sync) -2. [Supported Devices](#supported-devices) -3. [Getting Started](#getting-started) -4. [Device Setup](#device-setup) -5. [Using Sync Features](#using-sync-features) -6. [Book Matching](#book-matching) -7. [Managing Conflicts](#managing-conflicts) -8. [Best Practices](#best-practices) +## Supported devices today ---- +| Platform | Status | How it syncs | +| --- | --- | --- | +| **Web browser** | ✅ Fully supported | Live, as you read — the reader saves position and annotations continuously | +| **Android app** | ✅ Fully supported | The native client: progress, highlights, bookmarks, notes; reads offline and queues changes until you are back online | +| **KOReader** (Kindle, Kobo, PocketBook) | ✅ Fully supported | The Bookhoard KOReader plugin, over Wi-Fi | +| **Kobo, stock firmware** | 🚧 Server support complete | The native Kobo protocol is implemented and documented server-side; device-side enablement is the remaining piece — [Kobo Setup](devices/kobo-setup.md) has the details | -## What is Universal Sync? +Kindle and reMarkable are under consideration; KOReader already covers +several of those devices today. -**Universal Sync** is Bookhoard's cross-platform synchronization system that keeps your reading progress, highlights, and notes in sync across all your devices - automatically and in real-time. +## Getting a device into the circle -### Key Features +1. Connect the device to your server (each guide above walks its own path) +2. Approve the pending registration on the **Devices** page — the Android + app registers itself on first login; KOReader asks from its plugin +3. Done. Sync starts as soon as a device is approved -🔄 **Automatic Sync** - Your reading progress syncs automatically when you turn pages +The **Devices** page is mission control: see every registered device and +when it was last seen, remove a device (revoking its access instantly), and +work through **unlinked books** — device files that could not be matched to +a library book yet. -📱 **Multi-Platform** - Works with web browsers and KOReader, with native Kobo sync and mobile apps on the roadmap +## Book matching -📍 **Precise Location Tracking** - Supports EPUB CFI, page numbers, percentages, and character offsets +When a device reports a book, Bookhoard tries to match it to your library +by fingerprint, strongest first: -⚡ **Real-Time Updates** - See your progress update across all devices instantly +1. **Bookhoard UUID** — 100% confidence +2. **OPF UUID** — 95% +3. **SHA-256 hash** — 90% +4. **OPF identifier** — 85% +5. **ISBN / ASIN** — 80% +6. **Title + Author + file size** — 50% -🔒 **Secure** - No passwords on devices, web-based authentication only +Books that match automatically never bother you. The ones that can't +(side-loaded files, converted formats like EPUB → KEPUB, mismatched +metadata) land in the device's **unlinked books** list, where you can: -📴 **Offline Support** - Queue changes when offline, sync when reconnected +- **Auto-link** everything above a confidence threshold +- Review **suggestions** first, then link +- **Link by hand** — pick the right library book yourself ---- +## What syncs -## Supported Devices +**Reading progress** — page, percentage, chapter position, EPUB CFI, last +read time. EPUBs sync by percentage and CFI, PDFs and comics by page, so +the same book lines up on devices with different page counts. -### Currently Supported ✅ +**Highlights and notes** — the highlighted text, your note, its color, and +where it sits. Locations are stored in several forms at once (page, EPUB +CFI, percentage, character offset), which is why a highlight made on your +phone lands in the right place on an e-reader with different pagination. -| Platform | Status | Sync Method | Notes | -| ---------------- | ------------------ | --------------------------- | ---------------------------------- | -| **Web Browser** | ✅ Fully Supported | Real-time WebSocket | Any modern browser | -| **KOReader** | ✅ Fully Supported | Wi-Fi (Bookhoard plugin) | Kindle, Kobo, PocketBook hardware | +**Bookmarks** — location, title, and when you made them. -### Coming Soon 🚧 +Everything above is per-user and private: your devices see your data +because you approved them, other users see theirs, and revoking a device +ends its access on the spot. -| Platform | Status | -| --------------------- | ------------------------------------------------------------- | -| **Kobo Devices** | Native sync coming soon — use KOReader on Kobo hardware today | -| **Mobile Apps** | Android/iOS apps coming later | +## Conflicts -### On the Roadmap 🔭 +If the same book is read on two devices close together, Bookhoard does not +silently pick a winner — it records a **sync conflict** and shows it to +you: -| Platform | Status | -| --------------------- | ---------------------------------------- | -| **Kindle Devices** | Under consideration (no date yet) | -| **Remarkable Tablet** | Under consideration (no date yet) | +- The book's detail page grows a **Sync Progress** button (with a + "Progress conflict detected" notice) +- The **Conflicts** page (`/conflicts`) lists everything unresolved ---- +Resolving means choosing a strategy: keep one side's progress, keep the +server's current value, or let the **furthest progress win**. Until you +resolve, the conflict stays listed — nothing is lost, it just waits for +your call. -## Getting Started +## Offline reading -### Prerequisites +Devices keep working without the server. The Android app reads fully +offline and queues your changes, draining the queue when you reconnect. +E-readers push their progress on the next connection. The server holds a +per-device queue for inbound work and retries failures automatically; if +items pile up (a device offline for weeks), an admin can retry or clear a +device's queue from the server side. -1. **Bookhoard Server** - Self-hosted instance running and accessible -2. **Network Connection** - Devices must be able to reach your Bookhoard server -3. **User Account** - Created and logged in to Bookhoard web interface +## Reading history -### Quick Start - -1. **Set up your first device** - See [Device Setup](#device-setup) below -2. **Start reading** - Open any book in the web interface or on your device -3. **Turn pages** - Progress syncs automatically -4. **Switch devices** - Pick up any other device - your progress is there! - ---- - -## Device Setup - -For detailed device configuration instructions, see the appropriate setup guide: - -- **[KOReader Setup Guide](devices/koreader-setup.md)** - KOReader configuration (Kindle, Kobo, and PocketBook hardware) -- **[Kobo Setup Guide](devices/kobo-setup.md)** - Native Kobo sync (coming soon; use KOReader today) - -### Quick Overview - -**Registration Process** (KOReader): - -1. Install the Bookhoard plugin and enter your server URL in KOReader -2. Approve the pending registration on the Bookhoard **Devices** page (sidebar navigation) -3. That's it — sync starts automatically once approved - -**Device Management**: - -``` -Devices page (sidebar navigation) -``` - -You can: - -- View all your registered devices -- See last sync time and status -- Disable or remove devices -- View unlinked books that need matching -- Manage file aliases - ---- - -## Book Matching - -### What is Book Matching? - -When devices sync books, Bookhoard tries to automatically match them using: - -- **SHA-256 hash** (most reliable) - Content-based fingerprint -- **ISBN** - Standard book identifier -- **UUID** - Unique identifier from EPUB metadata -- **Title + Author** (fallback) - Last resort matching - -### Unlinked Books - -Sometimes a book on your device can't be automatically matched to your library. This happens when: - -- The book was side-loaded (not downloaded via Bookhoard) -- The file format was converted (EPUB → KEPUB) -- The metadata doesn't match exactly - -### Viewing Unlinked Books - -``` -Devices page → select device → unlinked books -``` - -### Resolving Unlinked Books - -1. **Auto-Link** - Automatically link books with 80%+ confidence -2. **Get Suggestions** - Review potential matches before linking -3. **Manual Link** - Choose the correct match yourself - -### Book Matching Priority - -1. **Bookhoard UUID** - 100% confidence -2. **OPF UUID** - 95% confidence -3. **SHA-256 hash** - 90% confidence -4. **OPF identifier** - 85% confidence -5. **ISBN/ASIN** - 80% confidence -6. **Title + Author + File Size** - 50% confidence - ---- - -## Managing Conflicts - -### Reading Progress - -**What Syncs**: - -- Current page number -- Reading percentage -- Chapter progress -- EPUB CFI (for EPUB files) -- Last read timestamp -- Reading position (viewport, zoom, scroll) - -**How It Works**: - -``` -You turn page → Device sends progress → Server updates database - ↓ - Broadcasts to all your devices - ↓ - Other devices update their display -``` - -**Supported Progress Types**: - -- **EPUB/MOBI**: Percentage + EPUB CFI + Chapter -- **PDF/DJVU**: Page number + Viewport position -- **CBZ/CBR**: Page number + Panel coordinates - -### Highlights & Notes - -**What Syncs**: - -- Highlighted text -- Notes and annotations -- Bookmark locations -- Colors and formatting -- Chapter/paragraph references - -**Universal Location References**: -All highlights are stored with multiple location types: - -- Page:offset (traditional) -- EPUB CFI (EPUB files) -- Percentage (0-100%) -- Character offset (exact position) -- Chapter references - -This ensures your highlights work across all devices, even with different page counts! - -### Bookmarks - -**What Syncs**: - -- Bookmark locations -- Bookmark titles -- Date created -- Reading position context - ---- - -## Sync Modes - -### Immediate Mode (Default) - -**Best For**: Normal reading, page turns - -**Behavior**: - -- Syncs every page turn -- Real-time updates -- Low latency -- Higher bandwidth usage - -**Recommended Settings**: - -- Auto-sync: ON -- Sync frequency: Every page turn - -### Checkpoint Mode - -**Best For**: Slow connections, battery saving - -**Behavior**: - -- Batches changes -- Syncs every 5 minutes or when connection allows -- Lower bandwidth -- Better for offline reading - -**Recommended Settings**: - -- Auto-sync: ON -- Sync frequency: Checkpoint mode - ---- - -## Offline Support - -### How Offline Sync Works - -1. **Offline Reading**: - - Read normally without connection - - All progress tracked locally - -2. **Queue Changes**: - - Progress changes queued automatically - - Annotations saved locally - - Up to 100 items in offline queue - -3. **Reconnection**: - - Connect to Wi-Fi - - Queue processes automatically - - Priority items first (book completion, notes) - - All devices updated - -### Offline Indicators - -**In Web Interface**: - -- Yellow status icon: Device offline -- Last seen timestamp -- "Pending sync" badge on books - -**On Devices**: - -- Sync icon: Gray = offline -- Sync icon: Blue = syncing -- Sync icon: Green = synced - ---- +Every reading session is logged — when, how long, on which device — and +feeds your [dashboard](dashboard.md) and stats. Your history is visible on +the **Progress** page and on each book's detail page, and it is yours +alone. ## Troubleshooting -### Sync Not Working +**Progress not updating** +Check the device is still on the **Devices** page and was approved; check +the server URL in the device's settings; confirm the device can reach the +server (open the server URL in the device's browser as a test). -**Symptoms**: Progress not updating across devices +**"Device not found"** +The device was removed or its token revoked — register it again and +approve it. -**Solutions**: +**A conflict keeps coming back** +Two devices are genuinely fighting over the same book. Decide which device +is the reader for that book, resolve with a strategy above, and avoid +simultaneous reading of the same title. -1. Check device is online: Devices page (sidebar navigation) -2. Verify sync is enabled for the device -3. Check sync URL is correct -4. Ensure device has network connection -5. Try manual sync: Open book → Menu → Sync Now +**High battery or data use** +That is the cost of continuous sync on a chatty device; e-reader plugins +let you space out their sync checks from the plugin's own settings. -### "Device Not Found" Error +## Device limits -**Cause**: Device not registered or authorization revoked - -**Solutions**: - -1. Re-register the device -2. Check device hasn't been removed -3. Verify correct device type selected - -### "Rate Limit Exceeded" Error - -**Cause**: Too many sync requests - -**Solutions**: - -1. Wait a few seconds -2. Switch to checkpoint mode -3. Contact admin to increase limits - -### Conflicts Detected - -**Symptoms**: "Sync conflict" notification - -**Cause**: Same book being read on multiple devices simultaneously - -**Solutions**: - -1. Open the book's detail page and click **Sync Progress**, or go to the Conflicts page (`/conflicts`) -2. Review both device progress -3. Choose which device's progress to keep -4. Or choose "Merge" (keeps furthest progress) - -### High Battery Usage - -**Cause**: Immediate sync mode with frequent page turns - -**Solutions**: - -1. Switch to checkpoint mode -2. Increase sync interval -3. Sync less frequently +Each account has a device limit — **10 by default**, and administrators can +raise or lower it per user. Removing a device you no longer use frees a +slot. --- -## Best Practices - -### For Optimal Performance - -✅ **DO**: - -- Use checkpoint mode when on slow connections -- Keep device firmware updated -- Use Wi-Fi when available -- Approve only devices you own -- Regularly check conflict resolution - -❌ **DON'T**: - -- Read same book on multiple devices simultaneously -- Ignore conflict notifications -- Register public/shared devices -- Exceed device limits (max 10 per user) - -### Organizing Your Library - -**For Best Sync Experience**: - -- Use consistent metadata (titles, authors) -- Avoid duplicate books in library -- Match files by ISBN when possible -- Use clean file structure - -### Managing Multiple Devices - -**Recommended Setup**: - -- **Primary Device**: KOReader on e-reader -- **Secondary Device**: Web browser (work/home) -- **On the go**: Web browser on a phone (dedicated mobile apps coming later) - -**Sync Strategy**: - -1. Read mainly on primary device -2. Check progress on web/secondary devices -3. Let auto-sync handle updates -4. Resolve conflicts promptly - ---- - -## Advanced Features - -### Conflict Resolution - -**Automatic Resolution**: - -- Most recent progress wins -- Timestamp-based comparison -- 5-minute window for conflict detection - -**Manual Resolution**: - -``` -Book detail → Sync Progress → choose winner -``` - -**Options**: - -- **Keep Device A**: Use this device's progress -- **Keep Device B**: Use other device's progress -- **Merge**: Keep furthest progress (combination) -- **Auto-Resolve Future**: Always prefer this device - -### Sync Queue Management - -**View Queue Status**: - -``` -Devices page → sync queue section -``` - -**Queue Stats**: - -- Pending: Waiting to sync -- Processing: Currently syncing -- Failed: Retry scheduled -- Completed: Successfully synced - -**Manual Actions**: - -- **Retry All**: Retry all failed items -- **Clear Queue**: Remove all pending items -- **Priority Sync**: Sync specific book immediately - -### Reading History - -**Automatic Tracking**: - -- Every sync session logged -- Time spent reading calculated -- Pages read tracked -- Device used recorded - -**View History**: - -``` -Progress page (sidebar navigation), or the book's detail page -``` - -**Privacy**: - -- Only you can see your history -- History kept for 365 days -- Exportable for backup - ---- - -## Security & Privacy - -### Device Authentication - -**Secure by Design**: - -- ✅ No passwords stored on devices -- ✅ Web-based approval required -- ✅ Unique tokens per device -- ✅ Revocable at any time -- ✅ Token encryption in transit - -### Data Protection - -**What We Store**: - -- Reading progress (page, percentage) -- Highlights and notes -- Device identifiers -- Sync timestamps - -**What We DON'T Store**: - -- Passwords on devices -- Reading content (your books) -- Unencrypted personal data -- Location data (GPS) - -### Access Control - -**Your Data**: - -- Only you can see your progress -- Admins cannot read your annotations -- Shared only with devices you approve - -**Device Access**: - -- Each device sees only your libraries -- Devices cannot access other users -- Revoking removes all access - ---- - -## Performance Tips - -### For Faster Sync - -1. **Use 5GHz Wi-Fi** - Faster than 2.4GHz -2. **Keep server nearby** - Low latency = better sync -3. **Regular updates** - Keeps firmware current -4. **Checkpoint mode** - For batch processing - -### For Better Battery Life - -1. **Checkpoint mode** - Fewer sync requests -2. **Increase sync interval** - Fewer updates -3. **Close when not reading** - Reduces background activity - ---- - -## FAQ - -### General Questions - -**Q: Does sync work with all book formats?** -A: Yes! EPUB, PDF, MOBI, CBZ, CBR and more. - -**Q: Can multiple users share a device?** -A: No, devices are tied to individual accounts for security. - -**Q: What happens if I delete a book?** -A: All sync data for that book is removed from the server. - -**Q: Can I export my reading data?** -A: Reading data isn't exportable from the UI yet — it's accessible via the API. - -**Q: Does sync work over the internet?** -A: Yes, if your server is publicly accessible with HTTPS. - -**Q: How much data does sync use?** -A: Approximately 1KB per page turn, 50KB per annotation. - -### Technical Questions - -**Q: How does sync handle different page counts?** -A: Uses percentage and EPUB CFI for universal positioning. - -**Q: Can I sync with Calibre anymore?** -A: Bookhoard's KOReader sync uses a dedicated plugin (server-side approval, device tokens) — no Calibre involvement required. - -**Q: What if I lose my device?** -A: Revoke it in settings and register a new one. - -**Q: Is sync end-to-end encrypted?** -A: Yes, HTTPS/TLS 1.3 for all sync traffic. - ---- - -## Getting Help - -### Documentation - -- **API Docs**: `/docs/api.md` -- **Device Guides**: `/docs/devices/` -- **Self-Hosting**: `/docs/install.md` - -### Support - -- **Issues**: Report bugs at GitHub Issues -- **Discussions**: Join GitHub Discussions -- **Email**: support@bookhoard.example.com - -### Community - -- **Forum**: community.bookhoard.example.com -- **Matrix**: #bookhoard:matrix.org -- **Discord**: discord.gg/bookhoard - ---- - -## Changelog - -### Version 1.0.x (2026) - -- ✅ Web sync support -- ✅ KOReader sync (progress, bookmarks, highlights, notes) -- ✅ Conflict resolution -- ✅ Offline queue -- ✅ Real-time WebSocket sync -- 🚧 Native Kobo sync (coming soon) -- 🚧 Mobile apps (coming later) - ---- - -**Last Updated**: August 2026 -**Version**: 1.0 -**License**: AGPL-3.0 +More detail on any device lives in its guide: +[Android](devices/android-app.md) · +[KOReader](devices/koreader-setup.md) · +[Kobo](devices/kobo-setup.md) · +[Calibre import](calibre-integration.md) · +and the [filter rules](filters/index.md) that power matching-adjacent +shelves like [smart collections](collections.md).