john-okeefe
d429534b12
Release / build-and-push (push) Successful in 2m24s
New media-items/deleted_annotations.md for the list/restore/purge endpoints; endpoint index updated. The KOReader protocol page documents deleted_highlights/deleted_bookmarks on the progress push and the deletion-propagation contract: keys learned only from server pulls, explicit arrays only (never absence), tombstone convergence via the metadata fetch, no resurrection from stale replays, and the web history as the restore path.
210 lines
7.9 KiB
Markdown
210 lines
7.9 KiB
Markdown
# KOReader Sync Protocol
|
|
|
|
KOReader uses a custom JSON-based sync protocol.
|
|
|
|
## KOReader Progress Sync
|
|
|
|
**Endpoint**: `POST /api/sync/koreader/progress`
|
|
**Auth**: Device token required
|
|
**Content-Type**: `application/json`
|
|
|
|
### Request Headers
|
|
|
|
| Header | Type | Required | Description |
|
|
| ------------- | ------ | -------- | ------------------- |
|
|
| Authorization | string | Yes | Bearer device token |
|
|
| Content-Type | string | Yes | application/json |
|
|
|
|
### Request Body
|
|
|
|
| Field | Type | Required | Description |
|
|
| ------------------ | ------- | -------- | ---------------------------------------------------- |
|
|
| library_id | string | No | Library UUID |
|
|
| books | array | Yes | Array of book sync data |
|
|
| books[].uuid | string | No\* | Book UUID (highest-confidence match; omitted on first sync of a newly downloaded book) |
|
|
| books[].sha256 | string | No\* | Full-file SHA-256 (64 hex chars); used to resolve the book when `uuid` is absent |
|
|
| books[].file_path | string | No | Device-local file path; used to create/look up a device file alias |
|
|
| books[].title | string | Yes | Book title |
|
|
| books[].authors | array | Yes | Array of author names |
|
|
| books[].progress | float | Yes | Progress percentage (0-1) |
|
|
| books[].percentage | float | Yes | Progress percentage (0-1) |
|
|
| books[].last_read | string | Yes | ISO 8601 timestamp |
|
|
| books[].chapter | integer | No | Current chapter |
|
|
| books[].epubcfi | string | No | EPUB CFI location |
|
|
| books[].character | integer | No | Character offset |
|
|
| books[].bookmarks | array | No | Array of bookmarks/highlights (shape, color mapping, and echo/dedup rules: see [Sync Bookmarks](../koreader/sync_bookmarks.md)) |
|
|
| books[].deleted_highlights | array | No | Highlights deleted on the device: `[{ "dedup_key": "..." }]` — keys previously served to this device (see [Deletion propagation](#deletion-propagation)) |
|
|
| books[].deleted_bookmarks | array | No | Bookmarks deleted on the device: `[{ "dedup_key": "..." }]` |
|
|
|
|
\* At least one of `uuid` or `sha256` should be present. The server resolves the
|
|
book through the shared `BookResolver` with this priority: `uuid` → `sha256` →
|
|
`file_path` alias → `title`/`author`. SHA-256 matching is **format-aware**: it
|
|
checks `media_items.file_sha256` first, then `media_item_formats.file_sha256`, so
|
|
a converted file (e.g. KEPUB or PDF) downloaded via OPDS matches even though its
|
|
hash differs from the primary format's hash.
|
|
|
|
### Example Request
|
|
|
|
```json
|
|
{
|
|
"library_id": "optional-uuid",
|
|
"books": [
|
|
{
|
|
"uuid": "book-uuid",
|
|
"title": "Book Title",
|
|
"authors": ["Author Name"],
|
|
"progress": 0.45,
|
|
"percentage": 0.45,
|
|
"last_read": "2026-01-30T20:00:00Z",
|
|
"chapter": 3,
|
|
"epubcfi": "epubcfi(/6/4/2:15)",
|
|
"character": 15432,
|
|
"bookmarks": [
|
|
{
|
|
"chapter": 3,
|
|
"datetime": "2026-01-30T19:55:00Z",
|
|
"notes": "highlighted text",
|
|
"pos0": "epubcfi(/6/4/2:15)",
|
|
"pos1": "epubcfi(/6/4/2:20)",
|
|
"page": 45,
|
|
"text": "highlighted text excerpt",
|
|
"type": "highlight"
|
|
}
|
|
],
|
|
"highlights": [],
|
|
"notes": []
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
### Response (202 Accepted)
|
|
|
|
```json
|
|
{
|
|
"sync_status": "accepted",
|
|
"books_synced": 1,
|
|
"conflicts": [
|
|
{
|
|
"book_uuid": "book-uuid",
|
|
"conflict_type": "progress_mismatch",
|
|
"device_progress": 0.45,
|
|
"server_progress": 0.42,
|
|
"resolution": "device_wins"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
## Book Resolution (UUID lookup)
|
|
|
|
**Endpoint**: `GET /api/sync/koreader/resolve?sha256={hash}`
|
|
**Auth**: Device token required
|
|
|
|
Read-only lookup mapping a file SHA-256 to the book's UUID (format-aware,
|
|
same `BookResolver` path as the progress push). Devices call this on the
|
|
first open of a newly downloaded book to learn the UUID **before** their
|
|
first pull. Full details: [Resolve Book](../koreader/resolve_book.md).
|
|
|
|
This matters for conflict avoidance: a device that pushes to bootstrap its
|
|
identity transmits its current (first-page) position, which the server
|
|
treats as a real progress update — overwriting/conflicting with genuine
|
|
mid-read progress from other sources. Resolve, then pull, then push.
|
|
|
|
### Example Request
|
|
|
|
```http
|
|
GET /api/sync/koreader/resolve?sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
|
|
Authorization: Bearer device-token
|
|
```
|
|
|
|
### Response (200 OK)
|
|
|
|
```json
|
|
{
|
|
"book_uuid": "book-uuid",
|
|
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
|
"title": "Book Title",
|
|
"author": "Author Name"
|
|
}
|
|
```
|
|
|
|
404 when no book in the library matches the hash.
|
|
|
|
## KOReader Metadata Fetch
|
|
|
|
**Endpoint**: `GET /api/sync/koreader/metadata/{book_uuid}`
|
|
**Auth**: Device token required
|
|
|
|
### Example Request
|
|
|
|
```http
|
|
GET /api/sync/koreader/metadata/book-uuid
|
|
Authorization: Bearer device-token
|
|
```
|
|
|
|
### Response (200 OK)
|
|
|
|
```json
|
|
{
|
|
"uuid": "book-uuid",
|
|
"sha256": "ff3e4501bf9d72dea2ae28731a6cb5b83d7a7532c05b5d2dd083d0dbc9193ebf",
|
|
"title": "Book Title",
|
|
"authors": ["Author Name"],
|
|
"progress": {
|
|
"percentage": 0.42,
|
|
"character": 15432,
|
|
"epubcfi": "epubcfi(/6/4/2:15)",
|
|
"chapter": 3,
|
|
"chapter_progress": 0.234
|
|
},
|
|
"annotations": {
|
|
"highlights": [...],
|
|
"notes": [...],
|
|
"bookmarks": [...]
|
|
},
|
|
"last_sync": "2026-01-30T20:00:00Z"
|
|
}
|
|
```
|
|
|
|
`sha256` is the canonical primary-format hash of the book on the server. It is
|
|
returned so clients can cache it regardless of how the book was originally
|
|
obtained. The library list endpoint (`GET /api/sync/koreader/library`) includes
|
|
the same `sha256` field on each book.
|
|
|
|
## Deletion propagation
|
|
|
|
The progress push is upsert-only: absence of an annotation from
|
|
`highlights`/`notes`/`bookmarks` is **never** interpreted as a delete (a
|
|
client with a category disabled must not wipe the server). Deletions are
|
|
reported explicitly:
|
|
|
|
- Devices remember the `dedup_key` of every annotation the server served
|
|
them (persisted locally, e.g. KOReader's sidecar `bookhoard_known_keys`).
|
|
- When one of those annotations no longer exists locally, the next push
|
|
lists its key in `deleted_highlights` / `deleted_bookmarks`.
|
|
- The server tombstones the matching rows (`deleted = TRUE`, kept for the
|
|
retention window). Tombstones are served back to *other* devices via the
|
|
metadata fetch's `deleted_highlights` / `deleted_bookmarks` arrays so the
|
|
deletion converges everywhere.
|
|
- A stale replay pushing the annotation's content cannot resurrect the
|
|
tombstone: device pushes carry no modification timestamp, so the save is
|
|
treated as older than the delete.
|
|
- Restoring is possible from the web book page's deleted-annotation
|
|
history (`GET /api/media-items/:id/annotations/deleted`, restore/purge
|
|
endpoints) until the retention window lapses.
|
|
|
|
Because keys are only learned from server pulls, a device-native annotation
|
|
deleted locally is simply never pushed again — it can never be mis-flagged
|
|
as a server annotation deletion.
|
|
|
|
## Book identification
|
|
|
|
Every client/sync interface (KOReader, Kobo, OPDS, the device-link UI, and any
|
|
future mobile app) resolves books through a single shared service:
|
|
[`internal/services/book_resolver.go`](../../../internal/services/book_resolver.go).
|
|
The import-time SHA-256 (stored on `media_items.file_sha256`, plus a per-format
|
|
hash on `media_item_formats.file_sha256` for KEPUB/PDF) is the canonical shared
|
|
identifier. New clients should resolve by SHA-256 via `BookResolver` rather than
|
|
re-implementing their own matcher.
|