Tonight's failures all traced to one blind spot: the converter could
only reason about text within a single block. A position at a chapter
heading sends walk-up context (heading + the paragraphs below, joined by
the plugin's block capture); a selection can span several paragraphs.
Neither shape could be verified (containment compared one block against
a multi-block quote, so the CORRECT structural landing at the heading
was rejected) nor matched by text search (it never crossed block
boundaries). The ladder then fell to the percentage rung — which labeled
its char-count guess Precision "exact" — and that confidently-wrong CFI
was stored: reading positions reopened paragraphs away from the true
spot, and a highlight echo overwrote the row's good web CFIs with a
garbage start anchor that made the highlight unpaintable ("disappeared").
Four changes, all in the forward converter and its consumers:
- Quote verification: after the structural walk lands, read the
whitespace-normalized document text forward from the landing point
(crossing block boundaries; inline spans join directly so drop-cap
splits still read as one word). A usable context must be a prefix of
that stream — which is exactly what device captures are: the text from
the position onward, or the selection between two anchors. The old
single-block containment checks remain as secondary acceptance.
- Cross-block text search: the search rung matches against the whole
document flattened in reading order, with every rune mapped back to
its source node and offset. A context spanning blocks now matches, and
the matched extent yields a true range end (EndEPUBCFI) that
highlights use as their end anchor, threaded through the facade as
CanonicalLocator.EndCFI.
- Honest labels: the percentage rung returns Precision "percentage" —
a char-count estimate must never masquerade as an exact anchor.
- Confident-only storage: progress adopts a converted locator solely at
structural/exact precision (section hrefs keep their legacy handling;
anything lower stores percentage only), and highlight conversion
returns CFIs only at structural/exact precision — a low-confidence
echo yields empty, which applyLWW coalescing turns into preservation
of the row's existing web CFIs instead of clobbering them.
Tests: walk-up context at a heading verifies structurally and lands in
the heading; a block-spanning context is found by search with a range
end landing in the following paragraph; the percentage rung is honestly
labeled; all drop-cap guards stay green.
Drop-cap markup like <p><span>C</span>onvergence of Heaven and Earth</p>
made getTextFromXPointer return just 'C'. The text search then matched
the first 'C' in the chapter and stored doc-start (/4/2/1:0) with
'precision: exact', so the web reader reopened at the chapter start
while the percentage looked mid-chapter.
- Add convertByStructuralPath: walk the parsed CRE ElementPath against
the raw XHTML (same-tag 1-based indexing, mirroring buildCREXPointer),
map CharOffset into the target element's text, and build the CFI.
Usable device text verifies the landing; disagreement falls through
instead of storing a confident-but-wrong CFI.
- Add usableContextText guard (>=8 runes, >=2 words): single chars can
never claim an exact text-search hit in either direction
(ConvertCREToStandard and reverseByTextSearch).
- Add drop-cap regression fixtures plus usable-context unit tests.
- Verified against the real book: DocFragment[26]/p[12]/span header now
converts to epubcfi(/6/52!/4/28/2/1:0) structural both with 'C' and the
full header, and round-trips back to DocFragment[26].
ConvertToCanonical/ConvertFromCanonical built a fresh CFIConverter
per call, and each annotation converts twice (pos0+pos1) — a book
with 200 highlights re-opened and re-parsed the EPUB 400+ times per
sync, and again per metadata pull. A bounded 8-entry cache keyed by
path now shares converters (the parsing work belongs on the server;
clients stay thin). CFIConverter gained a mutex around its lazily
built spine/doc caches since instances are now shared between
concurrent requests.
Adds CFIConverter.SectionPercentage: book-wide percentage for a CRE
xpointer from the spine char distribution (midpoint of its document)
— the server-side counterpart to dropping per-annotation
getPageFromXPointer lookups from the plugin.
When finding or extracting text in EPUB/KEPUB DOM trees, inline
formatting elements like <em>, <strong>, <i>, <b>, <span>, etc. should
not break text continuity. A reader sees 'Vokalia and Consonantia' as
one phrase regardless of the <em> wrappers around each word.
Add inline formatting element set and helper functions:
- isInlineFormatting: checks if an element is an inline phrasing element
- collectInlineText: flattens text across formatting elements within
a block-level parent, returning segments that map back to original
text nodes
- findBlockParent: walks up from a text node to find the nearest
block-level ancestor (used to scope text collection)
- findTextAcrossInlineElements: fallback for findTextInNode that
concatenates text within each block element (transparently crossing
formatting elements) and maps match positions back to actual nodes
- collectBlockElements: gathers all block-level elements containing text
The key invariant: text collection NEVER crosses block-level element
boundaries (<p>, <div>, <h1>-<h6>, <li>, etc.) to avoid concatenating
text from different paragraphs.
The findTextInNode function now tries single-text-node matching first
(fast path, unchanged), then falls back to cross-element matching only
when needed. This preserves performance for the common case.
Implements ConvertCREToStandard which converts CREngine XPointers
(e.g. /body/DocFragment[6]/body/div/p[47]/text().2399) to standard
epubcfi format (e.g. epubcfi(/6/12!/4/2[id]/4/1:7)).
Key components:
- indexChildNodes: faithful port of foliate-js's epubcfi.js algorithm
for computing CFI-compatible child node indices including virtual
positions, null positions between adjacent elements, and text chunks
- preprocessXHTML: converts XHTML self-closing tags (e.g. <a id="x"/>)
to open/close pairs so Go's HTML parser produces the same DOM as the
browser's XHTML parser
- buildCFI: walks up from a text node to body, computing CFI indices
at each level using indexChildNodes
- findTextInNode: regex-based whitespace-flexible text search for
context_text fallback positioning
- convertByPercentageOffset: estimates position via book-wide character
counts when no context_text is available
- ConvertCREToStandard: orchestrates text search → percentage fallback
Supports CREngine XPointer format, CREngine fragment ID format
(#_doc_fragment_N_anchor), and includes round-trip test coverage for
1984 and Crime and Punishment EPUBs.
john-okeefe