fix(sync): normalize character offsets to UTF-16 at the wire; refresh book offset on every verified save

Offset currency policy, now explicit: EPUB CFI terminals, CRE text()
offsets and the served char_offset handle are UTF-16 code units (the
EPUB CFI spec, and what foliate/readium/KOReader/Kobo clients actually
observe), while internal arithmetic — the book-wide character_offset
column and percentage fractions — stays rune-based, consistent with
TotalCharacters. For all-BMP books the currencies are identical, so no
stored value changes; astral-plane text (emoji, rare CJK) no longer
drifts.

Boundaries converted: resolveCFIToNode interprets incoming CFI terminal
offsets as UTF-16; textNodeAtUTF16Offset (née textNodeAtRuneOffset)
interprets CRE text() offsets as UTF-16; buildCFI and buildCREXPointer
emit UTF-16 terminals; blockCharOffset (the served char_offset) is
UTF-16.

Also fixes two character_offset column defects: heals wrote a BLOCK-
relative offset into the book-wide column, and verified-but-unhealed
saves (e.g. KOReader pushes) never refreshed it, leaving it stale
behind the anchor. VerifyProgressAnchor now returns the verified book-
wide rune offset and SaveProgress refreshes the column on every
verified save.

Tests: astral currency round trip (offset after an emoji must shift by
one unit between currencies, in both heal and exact-verify directions)
and book-offset ordering. The cmd/server/tests integration harness
failures under docker (library folder 400 during setup) reproduce on
the pre-change tree and are unrelated.
This commit is contained in:
John O'Keefe
2026-09-26 20:18:48 -04:00
parent aec226af1a
commit 8c3273a0fc
7 changed files with 273 additions and 61 deletions
+1 -1
View File
@@ -133,7 +133,7 @@ func writeTestEPUB(t *testing.T) string {
}
docs := []spineDoc{
{"doc1.xhtml", "<body><div><p>Chapter one opening page.</p></div></body>"},
{"doc2.xhtml", "<body><div><p>The family of Dashwood had long been settled in Sussex.</p><p>Their estate was large, and their residence was at Norland Park.</p></div></body>"},
{"doc2.xhtml", "<body><div><p>The family of Dashwood had long been settled in Sussex.</p><p>Their estate was large, and their residence was at Norland Park.</p><p>The family crest shows a globe \U0001F30D and a rocket \U0001F680 flying onward.</p></div></body>"},
{"doc3.xhtml", "<body><div><p>Chapter three contents.</p></div></body>"},
{"doc4.xhtml", "<body><div><p>Chapter four contents.</p></div></body>"},
{"doc5.xhtml", "<body><div><p>Chapter five contents.</p></div></body>"},