fix(sync): normalize character offsets to UTF-16 at the wire; refresh book offset on every verified save

Offset currency policy, now explicit: EPUB CFI terminals, CRE text()
offsets and the served char_offset handle are UTF-16 code units (the
EPUB CFI spec, and what foliate/readium/KOReader/Kobo clients actually
observe), while internal arithmetic — the book-wide character_offset
column and percentage fractions — stays rune-based, consistent with
TotalCharacters. For all-BMP books the currencies are identical, so no
stored value changes; astral-plane text (emoji, rare CJK) no longer
drifts.

Boundaries converted: resolveCFIToNode interprets incoming CFI terminal
offsets as UTF-16; textNodeAtUTF16Offset (née textNodeAtRuneOffset)
interprets CRE text() offsets as UTF-16; buildCFI and buildCREXPointer
emit UTF-16 terminals; blockCharOffset (the served char_offset) is
UTF-16.

Also fixes two character_offset column defects: heals wrote a BLOCK-
relative offset into the book-wide column, and verified-but-unhealed
saves (e.g. KOReader pushes) never refreshed it, leaving it stale
behind the anchor. VerifyProgressAnchor now returns the verified book-
wide rune offset and SaveProgress refreshes the column on every
verified save.

Tests: astral currency round trip (offset after an emoji must shift by
one unit between currencies, in both heal and exact-verify directions)
and book-offset ordering. The cmd/server/tests integration harness
failures under docker (library folder 400 during setup) reproduce on
the pre-change tree and are unrelated.
This commit is contained in:
John O'Keefe
2026-09-26 20:18:48 -04:00
parent aec226af1a
commit 8c3273a0fc
7 changed files with 273 additions and 61 deletions
+1 -1
View File
@@ -967,7 +967,7 @@ func (mh *MediaHandler) GetMediaReadingProgress(c *echo.Context) error {
}
if mediaItem, err := mh.db.GetMediaItem(c.Request().Context(), pgtype.UUID{Bytes: mediaUUID, Valid: true}); err == nil {
if path, perr := mh.libraryService.ResolveMediaPath(c.Request().Context(), mediaItem.LibraryID, mediaItem.FilePath); perr == nil && path != "" {
if finalCFI, sel, href, charOff, _, _, verr := wsync.VerifyProgressAnchor(path, progress.Epubcfi.String, contextText, pct); verr == nil && sel != "" {
if finalCFI, sel, href, charOff, _, _, _, verr := wsync.VerifyProgressAnchor(path, progress.Epubcfi.String, contextText, pct); verr == nil && sel != "" {
resp["css_selector"] = sel
resp["anchor_href"] = href
if charOff != nil {