bookhoard/docs/developer/api/progress/position-contract.md

Position Contract

How reading positions are represented, submitted, verified, and restored across clients. This is the contract every client speaks — readium-based apps, web (foliate), KOReader, and Kobo devices alike.

The server is the position authority. Every submission is independently verified against the book itself before it is stored; a client whose locator math is wrong cannot poison the stored position. In exchange, the server hands back handles clients can apply directly, so they never have to parse or trust CFIs themselves.

The three-tier submission

Clients submit what their renderer can reliably observe. All three tiers are accepted on PUT /api/media-items/:id/progress (and on the KOReader device endpoints):

Tier	Field	Currency	Notes
1	percentage	Float 0..1 of the whole book	Universal. The only field every client can supply.
2	context_text	Up to 100 chars, whitespace-normalized, starting at the anchor	The verification anchor. The server extracts the text at the submitted structural anchor and compares.
3	epubcfi	Standard wrapped CFI, terminals in UTF-16 code units	Optional structural anchor. KOReader submits a CRE xpointer and the server converts it.

Tier 2 is what makes the system self-correcting: percentages alone cannot distinguish "the reader's locator is right" from "the reader silently reported wherever it is currently scrolled".

Verification and healing (ingest side)

On every progress save for a convertible reflowable book that carries a context_text, the server:

1. Parses the submitted epubcfi (if any) and resolves it against the book's own XHTML.
2. Extracts the text at the resolved anchor and cross-checks it with the submitted context_text.
3. On a mismatch — or an unresolvable anchor, or no anchor at all — heals the position by text search, using the submitted percentage to disambiguate repeated phrases. The healed CFI and a recomputed percentage are stored in place of the submitted ones.
4. Refreshes the book-wide character_offset column from the verified anchor on every verified save, so it never goes stale behind the anchor.

If verification fails outright (book file unreadable, context not found and percentage cannot disambiguate), the error is logged and the submission is stored as-is — verification never rejects a save, it only corrects.

The restore handles

GET /api/media-items/:id/progress serves, alongside the raw stored fields, the server-derived handles for reflowable books with a resolvable anchor:

Field	Currency	Meaning
anchor_href	—	The spine document the anchor lives in (e.g. 1984.xhtml). This is how a client finds the right resource.
css_selector	—	Body-relative tag:nth-child(k) chain of the anchor's block element (e.g. body>div:nth-child(4)>p:nth-child(29)).
char_offset	UTF-16 code units	Offset of the anchor within the concatenated text of that block.
epubcfi	UTF-16 terminals	The stored canonical CFI — re-served healed if the GET-time re-verification improved it.
context_text	—	The stored verification context (≤100 normalized chars from the anchor).

Recommended restore sequence for a client:

1. Open the book and jump to the stored percentage (coarse floor).
2. Resolve anchor_href against your own spine (an ends-with match on document hrefs) and open that document if you are not already there.
3. Query css_selector in that document, walk its text nodes counting UTF-16 units to char_offset, and scroll that position into view.
4. If the anchor cannot be measured (renderer-specific laziness), the percentage floor stands.

SPINE NUMBERING HAZARD — read this before parsing a stored CFI

epubcfi spine steps index the OPF spine AS WRITTEN, including linear="no" items. Several rendering engines (notably readium) number their reading order EXCLUDING linear="no" items. When a book's cover (or any other item) is linear="no", the two numberings differ by a constant offset from that item onward — a client resolving a stored CFI's spine step against its own numbering lands in the WRONG DOCUMENT.

This is not hypothetical: "1984" epubs commonly have a linear="no" cover, which makes readium spine 0 = OPF spine 1. The server heals such numbering mismatches at ingest (that is what the context check is for), but the durable rule for client authors is:

Never resolve a stored CFI's spine step yourself. Resolve the document by anchor_href, then land with css_selector + char_offset. Treat the CFI as opaque server currency.

Offset currencies

Two counting systems are in play, and they are deliberately kept apart:

Quantity	Currency	Why
CFI terminal offsets (…/1:456)	UTF-16 code units	The EPUB CFI spec, and what every client observes (JavaScript .length).
char_offset (block-relative handle)	UTF-16 code units	Same reason — clients walk DOM text with JS semantics.
KOReader CRE text().N offsets	UTF-16 code units	crengine is UCS-16 internally.
character_offset (book-wide column)	Unicode runes	Internal, consistent with total_characters and the percentage derivations.

For all-BMP text the two currencies are identical. They diverge on astral-plane characters (emoji, rare CJK ideographs): one rune, two UTF-16 units. The server converts at every wire boundary; internal arithmetic never crosses.

context_text rules

• Starts at the anchor position (it may begin mid-word).
• Whitespace-normalized (all runs of whitespace collapse to single spaces).
• At most 100 characters.
• Comparison is containment-based (client and server suffixes of the same block verify in either direction); contexts shorter than 12 chars never match.

Engine-specific notes

• readium-based clients (the Android app): submit percentage + context_text + a CFI generated from the laid-out WebView. Their locally-generated CFI spine steps use readium numbering — the server heals the difference; nothing to do.
• Web (foliate): submit all three tiers; foliate CFIs are the same currency the server stores.
• KOReader: submits a CRE xpointer in the epubcfi field of the device payload (historical field name; it is a CRE xpointer, not a CFI). The server converts CRE → canonical at ingest and canonical → CRE on pull (koreader_xpointer in the metadata response).
• Kobo: submits kepub CFI locators, converted server-side the same way.
• Fixed-layout content (PDF/CBZ): the page index is the canonical locator; CFI/xpointer are meaningless and neither submitted nor served.