127 Commits
Author SHA1 Message Date
John O'Keefe a42c6d4f75 feat(scanner): force rescan = original scan; feat(filters): the manually-edited filter (faceted plan Phase 13)
Release / build-and-push (push) Successful in 3m7s
Force rescan contract change (owner-directed): the editorial-keep skip
is retired. Force rescan now re-derives EVERY field on EVERY book from
all sources — multi-file books included (the folder/zip flows ran the
full chain on create and on manifest-flagged edits, but skipped the
sidecar apply under force). metadata_overrides is the only protection:
ApplyMetadataOverrides re-applies manual values inside the writer, so
manual edits survive; everything else re-derives.

- A force pass also STAMPS each re-derived book's manifest with the
  current parser version (success-only — same truthfulness rule as
  Phase 10) and closes its open processing issues (Phase 11 self-heal).
  Force rescan and the ev-bump trickle now converge: either leaves the
  library fully current; the deploy order is bump-first, then Rescan
  All (documented in the plan and contract_pins).
- Single-file force rescan already ran the full chain — unchanged,
  now pinned.

NEW filter field manually_edited ("Manually edited", boolean):
cardinality(mi.metadata_overrides) > 0 / = 0. Composed-engine (qf)
only — the builder's Book facts group gains it with is true/false;
no value menu, no suggestion surface; family lens inherited.

Tests: multifile force applies sidecar metadata + stamps + override
protection; single-file pin; the filter both directions scoped to the
test library; adjacent suites (Audiobook, Calibre, ExtractorVersion,
CrossLibraryMove, ProcessingIssues, FilterBuilder, SchemaIdempotency)
green. The extractor mechanism itself is untouched — the owner bumps
scan_extractor_version manually via the admin UI (plan Phase 13 §3).
2026-10-20 22:05:42 -04:00
John O'Keefe 9b5defcb31 feat(issues): the honest processing-issues ledger — row-less import failures, self-healing resolution, family lens (faceted plan Phase 11)
processing_issues only recorded cross-library move rejections; files
that failed to import vanished into an error counter and stdout. The
ledger now records every failure class and heals itself:

Schema (in place, pre-release boundary):
- media_item_id NULLABLE + file_path TEXT (deliberately unbounded —
  an over-long path is itself a failure mode; a VARCHAR(500) ledger
  column died on the exact 22001 it existed to report — caught by the
  new test). Row-less failures key on (library, file_path); item-backed
  on (item, type). Widened dedup = COALESCE-expression unique index;
  the old table UNIQUE stays (NULLs pairwise-distinct for row-less).
- Conflict-target upsert: repeat failures UPDATE the same row and
  REOPEN resolved ones — never duplicates.

Wirings:
- CreateMediaItem failure (single-file + multifile createAudioBookItem)
  → row-less import_failed (error).
- Extraction hard-fail with fallback import → item-backed
  metadata_extraction_failed (warning).
- Change-path re-extraction failure (the ev/sidecar guard) → item-backed
  metadata_extraction_failed; LogProcessingIssue keeps its signature, a
  row-less helper joins it.

Auto-resolution:
- Successful import → ResolveOpenProcessingIssuesForPath (library,
  file_path).
- Successful (re)extraction → ResolveOpenProcessingIssuesForItem
  (single-file change path, multifile editorial apply).
- Unchanged-skip scans do NOT resolve: extraction never re-ran, so the
  degraded metadata is still degraded — the page heals exactly when the
  book does.

Family lens on ALL issues surfaces (list, stats, resolve, delete, SSR
page): explicit out-of-set library id → 403, never silent. This
tightens one pre-lens behavior — a nonexistent library id returned
200-empty, now 403 (pinned, changelog entry in docs/contract_pins.md).
List is LEFT-JOINed so row-less rows appear (media_item_id "",
title falls back to the path).

Tests: full lifecycle state machine through real scans (fail → appears,
repeat → no dups, success-at-path → resolved, LEFT-JOIN list shape);
family lens 403s with a fresh non-shared admin; all 7 pre-existing
issue suites green (two expectations widened to the lens, changelogged);
adjacent suites (CrossLibraryMove, Audiobook, Calibre,
ExtractorVersion, SchemaIdempotency) green; unit suites green.
2026-10-20 19:38:02 -04:00
John O'Keefe 91b0c45dd4 feat(scanner): extractor-version (ev) mechanism — parser fixes reach old rows without a force rescan (faceted plan Phase 10)
Parser fixes (7e4c97f6, genre model v2) never reached rows ingested
under an older parser: the sidecar never changed, so every fingerprint
said 'unchanged'. The manifest JSONB now carries the reserved 'ev'
stamp — the parser version that produced the item's row.

Semantics (all pinned by tests):
- ABSENT ev decodes as 1 ('v1-or-unknown'); stamping is truthful-only:
  create paths + post-extraction stores stamp the setting's current
  value; deploy-day backfill and the extract-failure guard stay UNSET
  (a row never claims a version its data can't prove — a mid-backfill
  bump self-heals next poll); repoint reconciliation preserves the
  stored ev (stat-only refresh re-extracted nothing).
- NEW setting scan_extractor_version (seeded + registry + getter,
  int default 1, category scanner): a manifest whose ev != the setting
  re-extracts through the SAME change path (full extraction →
  updateMediaItemWithMetadata → fresh stamped manifest). Sidecar-less
  books get an entries-less manifest as their ev home (create: stamped;
  first-sighting: unset).
- Catch-up bounded: evReextractBudgetPerScan=100, atomic (watcher pass
  can overlap a poll scan), reset per ScanFolders — amortized +
  resumable across polls; no force rescan, no flood.
- Multifile books included: the ev gate rides multifileSidecarChanged's
  shared ladder; ev-driven manifestChanged drives the editorial apply
  + stamped re-store.

Settings API: the unified GET/PUT /api/system/settings now lists the
key (registry-driven, auto-surfaces in the admin UI). Test harness:
the registry is now Loaded + wired into the SystemSettingsHandler
(main.go parity) — settings writes through the API were 503s before.

Verified: unit (ev normalization, budget CAS, GetExtractorVersion
cache paths, registry clamping); integration state machine —
create-stamp ev=1 → equality skips (corruption probe persists) →
wiped scan-state backfills UNSET without re-extract → bump re-extracts
ONCE (sidecar value restored, ev=2) → stable (second corruption
persists); multifile twin green; adjacent suites (Audiobook, Calibre,
CrossLibraryMove, SchemaIdempotency) green; live app restart re-scans
1198 files with 0 changed (deploy-day rule), setting listed at
/api/system/settings.
2026-10-20 18:24:08 -04:00
John O'Keefe 4ee24e7152 feat(filters): the rule builder — composed engines over the field registry (faceted plan Phase 9)
- Field registry (services/filter_registry.go): ~55 typed fields —
  every book column, file/lifecycle facts, the three HOLDING
  STRUCTURES (library, collections with excluded-aware membership,
  series with FULL primary+alternate_info membership), and the whole
  user-relative class (status, progress, last read, my rating, review,
  notes/highlights presence, note/highlight text, time spent, pages
  read, collections, added by) reading the VIEWER's rows. Operators
  are typed constants; negation is an operator.
- Composed engines (services/filter_query.go): one Go clause builder
  over raw pgx feeds the search (identical SearchMediaItemsUnifiedRow
  shape), the field-aware suggestions, and a true-COUNT live count.
  Mandatory WHERE head everywhere (family lens + integrity facts).
  Values are always bound parameters — no user string reaches a
  field/operator position, ever.
- Wire: the qf JSON param (validated strictly — unknown
  fields/operators/malformed values are 400); qf wins over flat params
  on hand-crafted URLs; flat deep links keep working unchanged.
- Endpoints: GET /api/media-items/filter-count (live count);
  field-values autocomplete upgraded in place — any registry field
  key, qf binds the OTHER rules (symmetric under-filter suggestions,
  zero-match values never offered); classic six without a tree keep
  their static pinned contract.
- UI: the drawer becomes the rule builder — the ONLY filter UI (templ
  markup, Alpine tree state, sanctioned island; grouped field menus,
  operator menus with red negations, field-aware value menus, live
  count, chips per rule with red negations, URL rehydration, smart
  serialization flat-when-simple else qf).
- Tests: composed semantics (AND/OR/negation/nesting/enums/bools/
  numbers/dates/empty), every user-relative field class with the
  my-data weld (two readers, different worlds), note-text substring
  pin (catharsis ≠ Carthage), family lens (hidden libraries nowhere,
  403s), suggestion-under-rules exactness, qf validation + precedence,
  flat-vs-tree equivalence; registry completeness unit test.
2026-10-20 14:49:50 -04:00
John O'Keefe ebb7f4a5b0 feat(filters): panel removal + All-Books field-values fix (faceted plan Phase 8)
- Remove the Phase 7 facet-panel UI: SSR route, facet_panel.templ,
  FacetCountsForView, drawer panel block, panel TS + .facet-* CSS.
  The ENGINE stays (FacetCountsUnderFilters, all-facets endpoint,
  drift guard) — it becomes the Phase 9 builder's machinery.
- Fix the All-Books bug: field-values autocomplete (author/genre/
  tags/narrator/series/language) treats library_id as OPTIONAL —
  absent aggregates every VISIBLE library (family lens in-query);
  explicit out-of-set library_id is now 403, never a silent pass.
  The four scalar Search*Values queries get the nullable shape.
- toast.ts: suggestion fetches are exempt from error toasts
  (console-only) — a failed background hint never interrupts.
- TestFacetPanelPartial deleted with the behavior; TestFieldValuesAll-
  Libraries pins the fix (200 + cross-library aggregation, family
  lens, 403). Contract-pins changelog entry added.
2026-10-20 12:20:51 -04:00
John O'Keefe f52bb6f073 feat(facets): all-facets panel with under-filter counts (faceted plan Phase 7)
The 9 Batch A facets could not be SET from the web UI, the facets
endpoint had no in-app consumer, and the registry's libraryTypes hints
rendered nowhere. This phase closes all three.

- ONE UNION-ALL sqlc query (FacetCountsUnderFilters) computes all 14
  listable facets in a single round trip: each arm omits its OWN
  facet's filter block and carries every other registered facet's
  block (copied mechanically from SearchMediaItemsUnified) — classic
  faceted semantics: with filters active each facet counts under the
  others; with none, blocks are neutral and the counts are the v1
  global-per-library ones. One query serves both cases. The 14
  per-column Batch C queries are deleted.
- Endpoint: no facet param → the new all-facets shape
  ({facets: {param: [{value, display?, count}]}}), filters bound via
  the shared ParseMediaSearchParams, library_id optional (the
  All-Libraries view, family lens still enforced); facet=X keeps the
  pinned single-facet shape (now routed through the same query with
  neutral filters).
- SSR partial GET /media-items/facet-panel: templ renders the drawer's
  value lists — registry order, Batch D library-type scoping
  (FacetsForLibraryType: manga_type on manga, reading_direction on
  manga+comics, narrator on audiobooks; All view = union), prettified
  genre labels + language display names, active values marked. Family
  lens: out-of-set explicit library_id is 403.
- Drawer wiring: the panel fetches on drawer open and after every
  apply with the CURRENT filter query string; a value click toggles
  that facet's comma-joined filter (hidden input where the drawer has
  no field — same as deep links) and re-applies the form; chips and
  URL stay in sync. Values are server-rendered (rendering doctrine);
  bookshelf.ts owns only the click/fetch orchestration.
- prettifyGenre moves to internal/utils as the canonical
  implementation (services' display enrichment shares it; templ wraps
  it; the TS port in bookshelf.ts is unchanged).
- Drift guard extended: FacetCountsUnderFiltersParams must carry every
  registry facet's param, the bools as pgtype.Bool.

Tests: under-filter exactness (own-block omission, OR-within,
AND-across), family lens (hidden values absent; 403 on both shapes),
all-shape + pinned-shape coexistence, partial rendering with Batch D
scoping + active marking + 403. Full container suite green except the
documented pre-existing TestAdminCreateUser red.
2026-10-20 08:24:36 -04:00
John O'Keefe 6152137156 fix(tests): repair stale pins surfaced by the Phase 6 full-suite run
Two suite failures, neither caused by scanner code (the Phase 6 diff
touches no handler):

- search_test invalid_library_id expected the pre-family-lens 404;
  7e586df4 shipped the AGENTS-mandated 403 for every out-of-set id
  (existence must not be enumerable) and docs/contract_pins.md:195
  documented it — this one subtest's expectation was missed. Pin
  widened to the shipped 403.
- TestWebSocketPingPong deleted: it asserted a text-level ping→pong
  exchange no server code has ever implemented and no client has ever
  sent; it passed vacuously through its read-timeout path until
  interleaved scan_progress broadcasts from the live server's scans
  reached the socket first. Keepalive is the standard protocol-level
  ping the hub already sends. Never-built behavior is not a pinnable
  contract.

Both recorded in docs/contract_pins.md. Full container suite is green
except the documented pre-existing TestAdminCreateUser red.
2026-10-19 22:33:08 -04:00
John O'Keefe 5245f292ca feat(scanner): quiet scans, enqueue coalescing, watcher probe, sibling cache
Part 2 of the faceted plan, Phase 5 (steps 5-6): the scan that used to
print ~4 lines per unchanged file now prints ONE summary line, and a
poll tick landing mid-scan no longer stacks a second full scan.

- Quiet scans (§4): the walk loop is silent — inserts, changes, and
  errors log inside processMediaFile; the duplicate
  'Successfully processed' branch (both branches printed it) and the
  per-file 'processing / already exists / same size, skipping' chatter
  are gone. Summary carries the trigger and a changed counter:
  'Scan of <folders> (trigger: poll): N files, N changed, N new, Xs,
  N errors'. Triggers threaded through every enqueue site: poll
  (startBackupScan), watcher (dirty-dirs flush), admin (scan API),
  initial (startup scan).
- Enqueue coalescing (§5): Worker.HasActiveScanForLibrary (JobResult
  gains additive job_type so only SCAN jobs coalesce — an import
  sharing the library id never blocks one). enqueueLibraryScan skips
  silently; the admin scan API answers 409 with the active job id.
  Enforces the one-scan-per-library invariant the bulk scan-state map
  leans on.
- Watcher capability probe (§4): ONE Add on a root folder; failure
  disables the watcher with a single startup line (replaces the
  per-directory warning spam on inotify-broken deployments). The
  watcher stays an optional latency reducer over scheduled scans.
- Per-directory sibling cache (§6): hasSiblingBookFileCached caches
  the ReadDir per directory for one scan — loose-image manga re-stats
  thousands of images sharing a handful of directories.

Tests: TestHasActiveScanForLibrary (unit — running/pending coalesce,
terminal jobs don't, non-scan jobs don't, empty library never, nil
worker safe). Full scanner suite green in the container.
2026-10-19 20:35:07 -04:00
John O'Keefe 6750e9e0dc feat(scanner): manifest-gated sidecar change detection + playlist sync on change path
Part 2 of the faceted plan, Phase 4 (steps 3-4). Size-only detection
was blind to metadata-sidecar edits; a size-stable book is changed by
EDITING its metadata.opf/metadata.json/cover — previously invisible
until a force rescan.

- Sidecar manifest fingerprints (media_scanner_manifest.go): the
  manifest stats exactly the names the extractors consume (metadata.opf,
  metadata.json, findSidecarCover patterns incl. <basename>.<ext>), so
  manifest and extractors can never disagree. Shape v1:
  {v, entries:[{n, s, m (unix nanos — second granularity would hide
  edits landing in the backfill scan's own second), h (sha256 — covers
  included: the repoint ladder needs a baseline hash for any stat-
  drifted sidecar; a hashless cover turns harmless mtime drift into a
  re-extraction, the exact flood this design kills)]}.
- Evaluation ladder in cost order: stat pairs identical → skip with
  ZERO bytes read; stat drift → hash that sidecar (equal = host repoint
  reconciliation, stat pair refreshed, no re-extraction; differing =
  changed); NULL stored manifest → backfill only, NO re-extraction
  (deploy-day rule: the first scan after release must not look like an
  mtime flood).
- Change path: re-read full row, extractMetadata once and pass it
  through updateMediaItemWithMetadata (preExtracted — metadata_overrides
  stay protected), run syncPlaylistCollectionsAfterCreate (Correction 2:
  a reading-order entry added to an EXISTING book now creates its
  collection; add-if-missing is idempotent), store a fresh full
  manifest. Creates seed their manifest at insert time.
- Multifile folder/zip books intentionally keep Phase 3 behavior: their
  change path does not consume sidecar extractors, so gating them would
  flip books 'changed' without picking anything up (plan correction —
  the plan's §3 multifile note assumed otherwise).
- UpsertMediaScanState + GetMediaScanState queries.

Tests: media_scanner_manifest_test.go (ladder unit tests — fast path,
repoint, same-size drift, appear/vanish, unhashable legacy entries);
TestScanManifestLifecycle (seeding, unchanged-skip stability, repoint
reconciliation with stat-pair refresh assertion, same-size edit
consumed, playlist sync on the change path + idempotency, no-leak
guard). Container runner.
2026-10-19 20:17:57 -04:00
John O'Keefe bbedbfb13f feat(facets+ui): clickable facet values everywhere, genre prettify, active-filter chips
Frontend half of the faceted-navigation Phase 2. 'Everything filterable
should be clickable': every displayed facet value links to the bookshelf
pre-filtered by its raw stored value; display forms are prettified at
render, never in the DB.

- templates/facet_links.templ: FacetLink (inline links) + FacetBadge
  (badge links) to /bookshelf?<facet>_filter=<value>; the raw value
  rides the param while the shown text is prettified, so matching and
  display can never disagree. Series/tags keep their dedicated pages
  by design; the card media-type icon stays unlinked (a derived kind,
  not a column value — deep-linking it would send wrong values).
- book_detail.templ: publisher, language, edition, genre, imprint
  cells + narrators line + age-rating/story-arc badges are now links.
- prettifyGenre (templates/utils.go + unit tests): title-cases only
  entirely-lowercase values; small words stay down except first
  position; & preserved. TS port for chip labels documents the twin.
- bookshelf.templ + bookshelf.ts: active-filter chips bar — Alpine
  renders chips from client-local form state (sanctioned island);
  one bubbled htmx:afterRequest hook keeps chips + shareable URL in
  sync; removeFilterChip clears the facet and re-applies. URL deep
  links now accept all 18 facet params (hidden inputs where the
  drawer has no field).

Pins changelog: Batch B/C entry in docs/contract_pins.md. Rendering
ownership guard (make lint-rendering) green.
2026-10-19 18:59:36 -04:00
John O'Keefe 7e586df48a feat(facets): facet registry, Batch A server params, genre model v2, facets endpoint
Phase 1 + backend of Phase 2 of the faceted-navigation plan. One
declarative facet registry (services/facet_registry.go) drives the
bookshelf filter system; a drift-guard unit test reflects over the
generated SearchMediaItemsUnifiedParams so a registry entry and its
sqlc.narg block can never disagree silently.

- Nine new filter params on SearchMediaItemsUnified: story_arc,
  imprint, edition (scalar-fuzzy), age_rating, format_group,
  manga_type, reading_direction (exact), explicit, abridged
  (presence-bound bools). Comma = OR-within multi-select on every
  text facet; AND-across-facets unchanged.
- Genre model v2 (user rule: a subgenre is not a genre, it's a tag):
  genre_lexicon.go rebuilt as shelf set + subgenre->shelf map;
  pickSmartGenre picks direct shelf > mapped subgenre > Fiction floor
  > empty (unknown-only stays tags-only). Type-words (Manga, Comics,
  Graphic Novel) join the demote list. Both 'adventure' and 'action &
  adventure' are shelves — provider data uses both and the verified
  traces pin them.
- Language filter UX: comma elements normalize through the ingest
  language map, so display names ('English') match rows stored as ISO
  codes ('en'). The language autocomplete now serves display names.
- NEW GET /api/media-items/facets?facet=X&library_id=... — registry-
  driven value+count listing (14 static per-column queries, family-lens
  join); language values carry display names; bool/range facets 400.
- Family-lens gates: an explicit hidden-library library_id is now a
  403 on the search endpoints (was a silent empty set) — JSON branch
  and HTMX grid partial both.
- handlers.ParseMediaSearchParams is the SINGLE param parser for the
  search JSON endpoint and the HTMX grid partial; the HTML branch
  previously dropped publisher_filter and narrator_filter silently.

Pins: docs/contract_pins.md changelog entries (additive request
params; new facets endpoint). Tests: facet_registry_test.go (shape +
SQL drift guard + library-type union), series_parsing_test.go genre-v2
cases, facets_integration_test.go (param filtering e2e, counts,
language display, 403s) — container runner.
2026-10-19 18:59:23 -04:00
John O'Keefe 7537063cb7 feat(filters+nav): author navigation end-to-end — arrays on every media response, deep-linkable bookshelf filters, clickable authors
Completes the A2c acceptance scenario: click an author on a book
detail page -> land on the bookshelf filtered to everything that
author wrote (including their share of co-authored books).

- Media detail + unscoped list resolved maps expose subtitle,
  explicit, authors, publishers (the library-scoped raw branch and
  the search endpoint already did — raw rows follow the SELECT).
- SearchMediaItemsUnified SELECT carries the new columns; the search
  endpoint serializes the row directly, so clients get them there too.
- book_detail.templ renders each author as a link to
  /bookshelf?author_filter=<name>.
- Bookshelf honors filter params in the URL on load (author_filter,
  series_filter, genre_filter, publisher_filter, tags_filter,
  narrator_filter, language_filter): pre-fills the form, applies it to
  the first grid load. Comma-joined values are multi-select via the
  server-side string_to_array split. Filtered views are shareable.

Pins widened: search rows (+7 keys), resolved keys (+4); changelog
entries in docs/contract_pins.md. Android follow-up plan written to
~/.opencode/plan/bookhoard-app-v2-compat.md.
2026-10-19 16:16:58 -04:00
John O'Keefe 1dbb0ce406 feat(collections): playlist auto-creation, position ordering, hide_finished, Up Next, dashboard eligibility
Phase 2b + 3 of the metadata overhaul (plan §2.4, §3.1-3.7).

Auto-playlist collections (§2.4): the scanner turns obviously-playlist
sidecar series (explicit '(Reading Order)'/chronological/publication/
in-order/timeline markers only — precision over recall) into curated
static collections: upsert by view_settings.imported_from_series then
name (rename-safe), position set ONLY on insert (clean integers; 1.5/1a
stay unpositioned rather than guessing), membership add-if-missing,
never removed or re-ordered by rescans, provenance in description +
view_settings, show_on_dashboard true (dashboard-eligible by design),
'scanner.auto_playlist_collections' setting (default on, Settings
Registry + schema seed).

Ordering (§3.1-3.2): collection listings order position ASC NULLS LAST
then added_at ASC (was added_at DESC — walked playlists backwards);
PUT /api/collections/:id/items/order applies full-order rewrites
(owner-only, service-gated).

hide_finished + Up Next (§3.3-3.4): GetCollectionItemsForDashboard
gains viewer_user_id + hide_finished (LEFT JOIN reading_progress,
COALESCE percentage < 0.999) and viewer_percentage rides the row for
card graying; GetCollectionUpNext serves the collection's reading-queue
head with the family-lens visible-library array enforced.

Dashboard (§3.5): user collections are dashboard-ELIGIBLE BY DEFAULT —
the show_on_dashboard gate (never writable for user rows since the
column shipped) is gone from GetUserCollectionsForDashboard; hiding
stays dashboard-prefs (hidden_collections) territory; hide_finished
filtered-everything now emits a Complete section instead of silently
vanishing; sections JSON carries is_playlist/complete.

Series page ordering hardening (§3.6): GetSeriesBooks tiebreaks by
title ASC — NULL/tied numbers previously returned heap (scan) order,
which is what the user saw.

Frontend custom-section-builder note: the builder has sent
show_on_dashboard: true since its creation; the API never bound it.
User-row visibility is now intentionally prefs-governed, so the field
stays unbound by design (documented in code).

Pins widened (docs/contract_pins.md): collection detail + up_next;
dashboard sections + is_playlist/complete; new reorder endpoint
recorded.

Verified: TestPlaylistCollectionsEndToEnd (new, full e2e: sidecar
series split, collection auto-creation with position 7, rescan
idempotency, owner-only reorder, finished-filter + completion marker +
Up Next drain) plus the full affected integration sweep green.
2026-10-19 15:25:40 -04:00
John O'Keefe 646477afad feat(schema): subtitle/explicit columns, authors+publishers arrays, float series numbers
Phase 1 of the series/collections/metadata overhaul
(~/.opencode/plan/series-collections-metadata-overhaul.md).

Schema (in place, idempotent):
- media_items.series_number INTEGER -> DOUBLE PRECISION: real-world
  ordinals are decimals (0.5, 1.5 novellas); INTEGER silently truncated
  them (calibre 2.5 pinned as 2 in tests).
- + subtitle TEXT, explicit BOOLEAN (sidecar ingest; ABS explicit /
  Audible isAdult).
- + authors/authors_search/publishers/publishers_search TEXT[] NOT NULL
  (narrators A2a precedent). Arrays are write truth; author/publisher
  varchars become trigger-regenerated display strings
  (sync_author_publisher_display) so every varchar consumer (trgm
  search, facets, API, OPDS/Kobo) keeps working unchanged.
- Row-level NULL normalization for the new arrays in
  set_library_type_name (A2 lesson: Go nil -> explicit NULL -> 23502).
- Backfills ARRAY[author]/ARRAY[publisher] — deliberately no
  comma-splitting ('Tolkien, J.R.R.' would corrupt; scanner kept only
  authors[0] historically).
- collection_items.position INTEGER + (collection_id, position) index
  for playlist ordering.

Go: sqlc regen; MediaMetadata/handler params gain Subtitle/Explicit/
Authors/Publishers (legacy singles still accepted, arrays win);
SeriesNumber int32 -> float64 end to end; metadata_overrides adapted
with legacy override keys ('author'/'publisher') preserved so existing
user overrides survive; bulk author edit writes the array.

Pins widened (docs/contract_pins.md): raw sqlc row key set + the six
new columns; changelog entry added.

Verified: unit suite green; integration sweep (bulk/smart collections/
scanner/stats/sync/queue/calibre/pins/audiobooks) green against the
shared dev DB after in-place migration. Pre-existing unrelated red:
TestOwnershipAndAdminTestSuite/TestAdminCreateUser timezone inheritance
fails identically at the pre-change commit (live shared state).
2026-10-19 14:17:50 -04:00
John O'Keefe ee98c0fb3b docs(dev): test-running policy + media-fixture rules; ignore testdata
Release / build-and-push (push) Successful in 3m29s
Integration tests require the containerized dev DB and bind-mounted
paths — a bare host `go test ./cmd/server/tests` fails on database
auth on every machine not running the compose stack (the old doc
prescribed exactly that flow). development.md now documents the
container runner and the iteration rule: run ONLY your own tests via
a filtered container invocation while working, and reserve full-suite
runs for pre-commit validation — several tests read live shared state
(system_settings, real library folders, the periodic backup scan's
websocket broadcasts) and flap on data or timing a change never
touched.

Binary media fixtures are never committed: internal/services/
testdata/ is gitignored by policy (the audio metadata tests read
real-format binaries that ffmpeg cannot synthesize — Audible freeform
atoms — so they live untracked on the machines that run the suite).
The new "Media fixtures" section carries the ffmpeg recipes for every
supported container, tag writing, embedded covers and ffmetadata
chapters, plus the manual end-to-end recipe for the live system
(one book per subfolder — a directory with ≥2 audio files ingests as
one folder book).
2026-10-19 07:54:55 -04:00
John O'Keefe 6216bd9bcf feat(audiobooks): multi-file books (folder-of-tracks + zip) + exhaustive metadata ingestion
Multi-file book ingest: a directory of ≥2 audio files (recursively —
disc-part folders like CD1/Disc 2 merge upward, siblings never merge)
ingests as ONE book whose file_path is the book-root directory; a zip
of audio tracks is its own book (file_path = the archive). Both write
the documented chapters abstraction {index,title,start,end} into
audio_metadata and add additive keys: layout ("folder"|"zip") and a
tracks[] manifest (per-track path, global-seconds start/end, duration,
size). Pre-grouping one-track strays under a book root are absorbed on
rescan (annotations cascade). Rescans take a stat-only fast path
(track count + total size + known durations unchanged → no re-probe —
a 1189-track book must not re-ffprobe every auto-scan); a zero-duration
book counts as changed so a ffprobe-equipped rescan fills durations in.
Folder-book roots count as present in the archive lifecycle pass, and
per-item rescan (RescanMediaItem) routes multi-file books through a
book-level refresh — the generic updater would read the root/zip as if
it were the book file and clobber format_group with "unknown".

Serving: GET /api/media-items/:id/chapters gains additive layout +
tracks keys (urls → the new endpoint). New GET
/api/media-items/:id/audio/:trackIndex streams one track — folder
tracks served from the real file (full Range via ServeContent), zip
tracks via a reopen-and-skip ReadSeeker over the entry (deflate has no
random access; acceptable at audio bitrates; stored entries stay
section readers). Both endpoints gate on library visibility (403 on
hidden libraries — family-lens rule). ServeFile and OPDS DownloadBook
refuse directory paths (folder books have no single-file download).

Web player (next commit) consumes the manifest; progress/bookmarks/
deep-links keep their global-seconds coordinates, so the server-side
progress model is untouched. sync learns FormatGroupAudiobook
(percentage passthrough in ConvertProgress); detectFormatType stops
reporting "unknown" for audio.

Metadata ingestion (Batch B) — read everything available, degrade
gracefully, documented precedence sidecars → folder structure →
embedded tags → filename: ffprobe tags are now a case-insensitive
lookup map (freeform keys pass through verbatim); composer fills
narrators (Audible's own convention, kept as ONE string — no reliable
multi-narrator delimiter); grouping + Audible freeform atoms
(SERIES/SERIES-PART/SUBTITLE/ASIN/ABRIDGED) fill series/index/
subtitle-join/ASIN/abridged (explicit sidecar abridged-false is
guarded); date/genre fall back to tags; album/album_artist/artist
fill title/author. Genre persists at create for all formats.
Embedded cover art (mp3 APIC, m4b covr, flac picture) extracts via one
optional ffmpeg call as a "<book>.cover.jpg" sibling — sidecar covers
win. CUE sheets (FILE-matched, INDEX 01 at 75fps) are the lowest-
priority chapter source under sidecars and embedded m4b chapters. A
lone m3u/m3u8 in a book root imposes playlist ordering + #EXTINF
names; (disc, track) tags reorder tracks when coverage is complete.
extractMetadata's default branch no longer pre-fills the filename as
title (tags must outrank it) — final filename fallbacks now exist on
BOTH the create and rescan paths (TestRescanKeepsFilenameTitle pins
that rescans cannot blank a title); tagless zip books take the zip's
own name, not their parent folder's.

Accepted containers widen to .m4b .m4a .mp3 .flac .zip .opus .ogg .oga
.aac .wav with real media types registered (audio/ogg per RFC 7845 for
Opus-in-Ogg) — all browser-playable; .wma deliberately excluded.

Tests: unit tests for CUE parsing, playlist hints, tag fallbacks
(synthetic ffprobe reports — ffmpeg cannot write Audible freeform
atoms), ordinal parsing, tag ordering, natural sort, grouping rules;
integration suite (11 tests) over the live dev DB: folder/zip ingest +
idempotency, stray absorption, chapters manifest, per-track streaming
with Range, hidden-library 403s, embedded covers + sidecar-wins, CUE
chapters, embedded m4b chapters, playlist ordering, tag fallbacks, all
five new containers with content-type assertions, rescan title pin.
Fixture media is real committed-format binary content living untracked
under internal/services/testdata/ (gitignored by policy; recipes in
docs/contributing/development.md → Media fixtures). Test image now
ships ffmpeg to match the runtime's audio toolchain.
2026-10-19 07:54:28 -04:00
John O'Keefe d5bb874651 feat(themes): Omarchy catalog, 12 light themes, Dark/Light Appearance picker
Expand the catalog from 11 themes (10 dark, 1 light) to 37 (24/13):
all 18 remaining Omarchy defaults (palettes from each theme's
colors.toml) plus 8 canonical light themes (Tokyo Night Day, Solarized
Light, One Light, Ayu Light, GitHub Light, Gruvbox Light, Everforest
Light, Kanagawa Lotus). Lupine leads the light list as the default
light theme; the app default stays tokyo-night.

Appearance panel: Dark/Light segmented filter (SSR-initialized from the
user's current theme), scrollable theme list, Bookshelf section pinned;
active row revealed on open. Light themes carry color-scheme: light and
the soft-shadow token set. Public-page theme selects get Dark/Light
optgroups. New moon/sun icons.

tailwind.config: safelist the new theme ids — Tailwind v3 purges custom
@layer base class selectors that appear in no scanned content file, so
an unsafelisted id silently ships without its CSS (found the hard way).

theme.ts: applyTheme now swaps only theme-* classes (changing theme
from the login select no longer wipes min-h-screen) and keeps the class
in sync on <html> for the new ThemeFlashGuard — a pre-paint inline
script on the four public pages that applies the saved theme before
first paint.

Kanagawa's UI accent uses crystalBlue (#7e9cd8) instead of Omarchy's
foreground-as-accent so links stay distinguishable from text.
2026-10-18 12:08:07 -04:00
John O'Keefe 45b8542e25 fix(api): honor the request's origin as the writer label — app rows stop wearing "web"
Release / build-and-push (push) Successful in 3m36s
Every web-API write endpoint hardcoded Source:"web", so the Android
app's own pushes landed with last_sync_source/last_modified_source
"web" — the app was indistinguishable from the web SPA in progress
conflicts, device_sync_data, and annotation bookkeeping. The app CAN
say who it is (bookmark create already carried origin → origin_source);
the server just ignored it for labeling.

All six write sites (progress PUT, bookmark/note/highlight create+PUT)
now parse an ADDITIVE optional "origin" key, normalized through one
gate: "android"/"ios" pass through, anything absent/unknown stays
"web" (the label keys conflict claims and device_sync_data — free-form
client strings would pollute them). Precedence on progress: a paired
X-Device-Token still outranks origin. Bookmark origin_source now rides
the same gate. The web SPA sends no origin — its rows are unchanged;
KOReader/Kobo handlers keep their own labels.

Wire discipline: additive request keys only (rule 2), response shapes
untouched, pins changelog widened in-commit, locked by
TestOriginWriterAttribution (android lands, default stays web, unknown
normalizes). Full battery: only the two known pre-existing failures
(TestOPDSLibraryFolders rot, TestWebSocketPingPong — both reproduce
with the change stashed).
2026-10-17 21:44:27 -04:00
John O'Keefe 010c0cb287 test(contracts): whitelist the unified row's two new keys on search rows
The H4 row-completion commit made abridged/audio_metadata emit on the
search wire (the endpoint serializes the row directly) — exactly what
the pin exists to surface. Whitelisted per the additive rule with the
changelog entry.
2026-10-17 16:57:20 -04:00
John O'Keefe 7d698ea0d8 fix(annotations): bookmark and note PUTs land through the sync service (LWW by id)
H3 census bugs #2 and #3: both web PUTs used plain queries that
bypassed the sync-aware layer entirely — no LWW, no source bookkeeping,
and locator fields the client couldn't even send. The highlight PUT was
the only one already routed (via HighlightID); the other two now mirror
it:

- SaveBookmarkRequest gains BookmarkID, SaveNoteRequest gains NoteID:
  id-targeted resolution reads the exact row, verifies ownership,
  LWWs under its stored dedup key, and resurrects tombstones when the
  edit is newer (same doctrine and comment structure as the highlight
  path).
- Fields the PUT omits are coalesced from the stored row before the
  full-column sync update — the legacy web PUT sends only
  title/notes/position, and zeroing cfi/percentage/page on every edit
  would have been data loss. Verified both directions by
  TestH3_BookmarkPUTPreservesLocators (omitted survives, explicit
  lands).
- The bookmark PUT's wire DTO gains the create endpoint's locator keys
  as optional fields (documented additive request keys in
  contract_pins.md); the note PUT keeps its content+position wire.

UpdateMediaBookmark/UpdateMediaNote plain queries are now uncalled
(the generated queries stay until the H4 query sweep).
2026-10-17 15:53:12 -04:00
John O'Keefe ff029f1ad5 fix(annotations): note LWW re-save 500'd — UserID missing from the sync update params
Pin-first batch for H3: freeze web annotation CRUD response shapes
(create 201 / update 200 / delete 204; full sqlc row models) before
the stack consolidation touches anything.

The new pins immediately caught a live server bug the 28 unit tests
could never see (annotations_test.go is pure-unit: keys and compare
logic, no DB): applyNoteLWW built UpdateMediaNoteForSyncParams without
UserID, so the query's WHERE id=$1 AND user_id=$13 compared against
zero — every dedup-hit note re-save (same content+position) returned
500 "update note: no rows in result set" and the update never landed.
The bookmark twin passes UserID and works; the note stack is the clone
that dropped the field.

Fix: pass UserID in the params (one line). Pins go green and become
the regression net; changelog records the new surfaces.
2026-10-17 15:38:47 -04:00
John O'Keefe de9e66636a docs(agents): test-failure triage discipline — suspect the test first, but test edits stay asymmetric
2026-10-17 lessons from the hardening session: most test/bruno failures
were harness/schema/environment drift (drifted body keys, truncated
payloads, container-only paths), found by triaging the test before the
server. Codified as: triage order (test schema -> environment -> cheap
probe), with the guard against "writing tests to work" — test/contract
edits are asymmetric with code fixes: only harness plumbing fixes,
additive pin widenings with a changelog entry, or deletions of tests of
removed behavior. Pins were green at a known commit; git history
arbitrates stale-vs-changed. A right-looking test that fails is prima
facie a server bug: stop and flag.

contract_pins.md rule 4: narrowing/removing a pinned key requires the
user's explicit sign-off — a pin freezing buggy behavior is a ledger,
not a license.
2026-10-17 12:39:59 -04:00
John O'Keefe 28bd323505 refactor(progress): SaveProgressRequest becomes the canonical PUT payload; GET serializes from the row
Batch H1 steps 1-2 of the hardening plan. Wire contracts frozen; the
pinned shapes are unchanged modulo the whitelisted additive GET keys
(docs/contract_pins.md).

1. One canonical request struct. The web PUT binds SaveProgressRequest
   directly (identity/attribution fields are json:"-" so they stay
   server-owned) and copies its server fields once — the hand-rolled DTO
   and the 20-line field-copy block are gone. Parse set for existing
   clients is IDENTICAL to the old DTO (verified in web/src: the ebook
   reader sends percentage/current_page/total_pages/reading_mode/epubcfi/
   context_text/chapter/zoom_level — reader.ts:3113-3143; the audiobook
   reader sends percentage/chapter/audio_offset_seconds/elapsed_seconds/
   reading_mode — audio.ts:518-520).

2. GET /api/media-items/:id/progress now serializes through one
   row-struct-driven builder instead of a hand-map, so a row column
   cannot be silently forgotten again (the audio_offset_seconds
   omission). Existing keys are verbatim; the row already carried
   audio_offset_seconds/reading_mode/viewport_x-y/zoom_level/
   scroll_position_x-y/panel_number and they surface as WHITELISTED
   additive keys. The web audiobook reader already reads
   audio_offset_seconds (audio.ts:219) — with the key absent,
   durationless books resumed at chapter start (offset always 0); they
   now resume correctly. The conditional context_text bolt-on and the
   server-derived css_selector/anchor_href block are unchanged —
   restore precedence stays percentage-first, never re-derived.
2026-10-16 17:22:32 -04:00
John O'Keefe a2ea34232c test(contracts): pin wire shapes before the canonical-types migrations
Batch H0 of the hardening plan: freeze the JSON key SETS of every
load-bearing surface so H1-H7 can only change them additively.

- cmd/server/tests/contract_pins_test.go: 14 tests / ~30 subtests pinning
  progress GET+PUT, the KOReader plugin surfaces (progress push/pull,
  library, bookmarks), the Kobo adapter shapes (initialization, markup,
  sync-from-server), collections detail all four branches + preview,
  series JSON, search rows, dashboard sections, media list BOTH branches
  (raw sqlc rows vs resolved 38-key map), and SSR smoke for
  bookshelf/collections/series/tags.
- docs/contract_pins.md: the pinned-shape inventory + the rule that a
  batch widening a shape must widen the pin and this doc in one commit.

Suite is green on the unmodified tree at b4ae423. KOReader/Kobo/progress
pins are the hard freeze (the plugin is an external client).
2026-10-16 16:02:14 -04:00
John O'Keefe ce366b1604 feat(stats): Library tab — per-library inventory stats + audio-aware Reading tab
The /stats page gains a second tab (Reading | Library) implementing the
Audiobookshelf-style library dashboard (HANDOFF item 9): media_items
INVENTORY aggregates — items, hours/pages, distinct authors, size,
distinct narrators, Top Genres / Top Authors bars, Longest / Largest
items — for one library or the All-libraries aggregate.

- GET /api/stats/library-usage?library_id=<uuid|omit>; family lens
  enforced: a library_id outside the requester's visible set is a 403,
  never a silent pass (AGENTS.md standing rule; 431fec0 history)
- six sqlc queries with the visibility pattern + healthy-item filter
  (archived_at IS NULL AND missing_scan_count = 0); hours_mode lets the
  client swap the Hours/Pages card and the Longest ranking per scope
- Reading tab unit audit (HANDOFF item 3, settled 2026-10-16): minutes
  are the merged currency everywhere honest; only the PAGES unit splits
  — Pages card goes mode-aware (Pages · text / Listening Time in an
  audio-only pool) and the charts row gains Listening per Day (hrs)
  when audio minutes exist in the range; summary daily buckets gain
  audio_minutes split by the item's library type
- tab state in the URL (?tab=library&library_id=…); selector fed by the
  existing /api/libraries/visibility endpoint; main.js?v=4
- integration tests: family-lens 403 pin, hidden-library exclusion
  (differential), seeded text+audio aggregate correctness, auth gate
- docs/developer/api/stats/get_library_usage.md

Known-remaining suite failures are environmental: TestConflictList_Empty
+ TestWebSocketPingPong collide with the concurrently-running app session
(live emulator pushes + scans on the shared dev DB), and
TestAdminCreateUser's timezone expectation drifted from live system
settings — all three fail on the clean tree too.
2026-10-16 10:20:17 -04:00
John O'Keefe f42720ac64 docs: retire PROJECT_GUIDELINES.md — library-visibility rule now in AGENTS.md + dev docs
PROJECT_GUIDELINES.md had drifted out of reality (podman-only rule
while the stack runs docker; verifier scripts wired to Makefile targets
nobody runs). Trashed it, its two verifier scripts, and the Makefile
targets; the rules that still matched reality (docker runtime, schema
in place until release, server binary only via Docker, services layer,
integration-test pool rules) are absorbed into the new AGENTS.md.

The library-visibility family-lens rule now has a real home:
- AGENTS.md: the standing rule agents auto-discover (new media
  endpoints MUST thread the visible-library set; 403, never silently
  pass; history of the 2026-10-13 leak).
- docs/developer/library-visibility.md: the full explanation (helpers,
  SQL pattern, per-item content gates, series per-library), linked
  from the developer docs index.
Stale doc links updated (docs index, contributing, development).
2026-10-13 20:18:42 -04:00
John O'Keefe 72839921f2 drop DOC and LIT from the pipeline (deliberately unsupported)
.IMAGE  — extensions removed from AllowedExtensions (scan gate),
bookExtensions, MimeTypes and sync/format.go maps: .doc/.lit files
are no longer scanned or indexed. classifyFormatGroup gains the
RTF and PDB reflowable arms that were missing when those rungs
shipped (rows only reclassify on creation). Scanner docs updated.

DOC + LIT have no credible JS tooling (mammoth is docx-only; LIT
needs LZX and its DRM variants are dead) and text-extraction-only
support would misrepresent what the reader can do. Verified: probe
.doc/.lit files watched but never scanned; probe .txt control
scanned; TestClassifyFormatGroup extended and passing.

Docs change granted in-session by the user (extensions + format
docs).
2026-10-03 16:11:04 -04:00
John O'Keefe 0be70f0051 docs: drop AZW3 from documented format expectations
AZW3/KF8 is a proprietary legacy Amazon format (the pipeline moved to
KFX in 2015; KDP dropped MOBI-family uploads in 2022) and does not
render in the web reader. The scanner still ingests .azw/.azw3 files
(code unchanged) — they are simply no longer documented as a supported
expectation.
2026-09-30 21:22:44 -04:00
John O'Keefe b4b1deafb8 docs(user): Android app guide — LAN setup and the Android 16+ local-network permission
Android 16+ gates per-app access to local networks behind a runtime
permission; without it the app's LAN logins time out with zero packets
leaving the phone (browser works — it is not per-app-gated). The new
device guide covers the sideload install, first-run server URL, the
permission prompt (and manual re-grant path), and a triage table for
the login screen's error line. Per the user: overlay-VPN addresses are
not a consideration and stay undocumented.
2026-09-29 18:12:57 -04:00
John O'Keefe b10bf3e8c7 docs(api): real progress endpoint contract; remove nonexistent-endpoint docs
The progress documentation described GET/POST /api/progress/:id —
endpoints that do not exist in the router — while the endpoint every
client actually uses (GET/PUT/DELETE /api/media-items/:id/progress) had
no field-level docs at all.

New:
- progress/position-contract.md: the canonical position model — server
  as position authority, the three-tier submission (percentage /
  context_text / epubcfi), ingest verification and healing, the restore
  handles, the OPF spine numbering hazard (canonical CFI spine steps
  include linear="no" items; clients resolve documents by anchor_href
  and land by css_selector + char_offset, never by spine step), the
  offset currencies (UTF-16 at the wire, runes internal), and
  context_text rules.
- progress/get_media_progress.md and update_media_progress.md: the real
  endpoints with full field tables, conditionality of the restore
  handles, the first-page anti-clobber guard, and healed-response
  semantics.
- progress/delete_media_progress.md: the real DELETE route.
- koreader/sync_progress.md and koreader/get_metadata.md rewritten to
  the actual payloads: the plugin sends a single-book array whose
  "epubcfi" field is a CRE xpointer; the metadata response navigates
  via koreader_xpointer (canonical CFI converted back to CRE), with
  page as the canonical locator for fixed-layout books.

Removed: the five files documenting the nonexistent /api/progress/:id
GET/POST/DELETE endpoints. Kept get_progress_history.md (that route
exists). The legacy developer/api-reference.md and the indexed
api/api-reference.md progress sections now match the wire and link the
new docs; the duplicate "Universal Progress" section points at Reading
Progress.
2026-09-26 21:27:44 -04:00
John O'Keefe b4c956aed4 docs(user): web reader guide — navigation, touch selection, highlights
User-facing documentation for the rebuilt web reader (§6.5 of the app
handoff): swipe-to-page / tap-for-menu touch navigation, long-press word
selection with drag extension and handles, the selection popover (color
dots, notes with save-together-on-create, copy incl. plain-HTTP LAN,
edit/delete by tapping a painted highlight), below-the-selection popover
placement rationale, PDF/comic behavior, and annotations drawer.
Linked from the user documentation portal and the docs index quick
links / quick-find tables.
2026-09-17 20:13:45 -04:00
john-okeefe 9e516b96cc docs(android): require reader settings parity with the web reader
The web reader's font roster (Literata default, plus seven self-hosted
variable fonts), typography controls, chrome/reading theme split,
fx brightness/contrast/invert stack, tap zones, and highlight palette
are the reference design for the Android reader - only the mobile
presentation differs. Document the mapping to the synced reader_settings
model so the app reuses it instead of inventing a parallel one.
2026-08-29 15:25:52 -04:00
john-okeefe 44735554f7 docs(api): document the real book download route
The documented GET /api/media-items/:uuid/download is not registered
anywhere - MediaHandler.DownloadBook exists but no route mounts it.
Book files (and covers) are actually served by the JWT-authenticated
GET /uploads/library-{id}/{path} route that the web reader uses.

Rewrite the download doc around the real file route (URL construction
from the item's library_id and relative file_path, MIME/Cache headers,
error codes), note the dead handler so nobody relies on the phantom
endpoint, and correct the API reference index. Mention the OPDS device
route as the conversion-capable alternative.
2026-08-29 10:25:59 -04:00
john-okeefe 311049379d docs(android): add QR pairing sign-in to roadmap, update auth design
Document the authentication decision reached for the Android client:
username/password login is primary (the app needs the user-JWT API
surface that device tokens cannot reach), with the app self-approving
its own device registration post-login so it still shows up on the
Devices page with sync attribution.

Add the Netflix-style QR pairing flow to the post-v1 roadmap with its
constraints: the QR grants a full login with zero typing; a typed-code
fallback covers phones with broken cameras; KOReader keeps its existing
flow (no typed codes there); and pairing must encode the configured
BASE_URL rather than a detected LAN IP so remote instances
(https://public.domain) work identically.
2026-08-28 22:45:29 -04:00
john-okeefe f65db5ab4f docs(api): align auth/devices/libraries/media-items docs with handlers
Verified against the Echo routes and handler structs, fixing drift that
would break API clients:

- login: response field is access_token, not token (AuthResponse struct)
- register status: status is only pending|approved; expiry is HTTP 410
  (not a status value), approved responses are single-use, and pending
  registrations do not survive server restarts
- visible libraries: endpoint is GET /api/libraries/visibility and
  returns a top-level array of full library rows, not a wrapped object
- media items list: response is {"data": [...]}, library_id is optional,
  limit defaults to 50 (max 1000), no total field; document the sort
  parameter, the two response shapes, and raw-vs-resolved file paths

refresh and device-registration docs verified accurate; no changes.
2026-08-28 22:15:42 -04:00
john-okeefe b927ed9988 docs(android): add native Android client design doc
Document the planned native Android client (bookhoard-app): product
vision, tech stack and rationale (Kotlin + Compose + Readium over
hybrid/Flutter/KMP alternatives), module architecture, offline-first
sync flow over the existing REST/WebSocket API, reader and comics/manga
UX, iOS posture, distribution and licensing, and a five-milestone
roadmap. The client is a thin, offline-first consumer of the server's
existing device registration, universal progress, annotation, and
conflict-resolution APIs — no server changes required.
2026-08-28 20:59:31 -04:00
john-okeefe e2953c4a01 chore(license): relicense project from GPL-3.0 to AGPL-3.0
Replace the GPL-3.0 license text with the full GNU Affero General
Public License v3.0 text, strengthening copyleft coverage for the
network-service use case (users interacting with Bookhoard over the
network are entitled to the corresponding source).

- LICENSE: swap GPL-3.0 text for the canonical AGPL-3.0 text (gnu.org)
- README.md: update both license references (Project Status and
  License sections) from GPL-3.0 to AGPL-3.0
- docs/user/sync-guide.md: update the footer license reference

The bundled BSD 3-Clause license in internal/sevenzip/LICENSE is a
third-party dependency license and is intentionally left unchanged.
2026-08-28 20:12:20 -04:00
john-okeefe d429534b12 docs(api): deleted-annotation history + KOReader deletion propagation
Release / build-and-push (push) Successful in 2m24s
New media-items/deleted_annotations.md for the list/restore/purge
endpoints; endpoint index updated. The KOReader protocol page documents
deleted_highlights/deleted_bookmarks on the progress push and the
deletion-propagation contract: keys learned only from server pulls,
explicit arrays only (never absence), tombstone convergence via the
metadata fetch, no resurrection from stale replays, and the web history
as the restore path.
2026-08-22 13:16:54 -04:00
john-okeefe 91c8be8562 docs(api): document the KOReader resolve endpoint
Add koreader/resolve_book.md for GET /api/sync/koreader/resolve, list
the endpoint in the API reference, and describe the resolve-then-pull-
then-push linking flow in the KOReader protocol page — including why a
device pushing to bootstrap its identity creates progress conflicts for
books already mid-read from other sources.
2026-08-22 09:57:44 -04:00
john-okeefe 078c4b1f3f docs(api): add system/hash-conflict sections to API references
Monolithic api-reference.md:
- New 'System Settings & Configuration' and 'Hash Conflicts' sections
  (endpoints, examples, response shapes) with TOC entries
- Device Management: add the sidecar config/download endpoints
- Fix stale registration flow: correct auth_url path, drop phantom
  device_id, add poll_interval/setup_instructions, status endpoint is
  POST /api/devices/register/status, and sync_endpoints point at
  /api/sync/koreader/*
- Mark PUT /api/libraries/scan-settings as legacy/superseded
- Repair Additional Resources and Collections links (dead
  COLLECTIONS_API.md / KOBO*_SETUP.md / missing-guide references)

Split api-reference.md index:
- Quick links and sections for System (settings + config) and the
  admin hash-conflict endpoints; device sidecar endpoints under Device
  Management; browse + legacy scan-settings routes under Libraries
2026-08-20 14:41:05 -04:00
john-okeefe c49a9605ff docs(api): rewrite KOReader bookmark sync for current protocol
sync_bookmarks.md documented a request shape the handler never accepted.

- Document the real body: book_uuid/book_sha256 (either required,
  SHA-256 is format-aware), plus separate bookmarks/notes/highlights
  arrays using the shared KOReader annotation shape (pos0/pos1, page,
  text, type, per-annotation book_sha256, dedup_key, percentage)
- Document color semantics from 178fb2e/dafcadd: KOReader palette
  names map to web hex swatches at the boundary, echoes carry no color
  so stored web colors survive round-trips, explicit colors are device
  edits
- koreader-protocol.md: cross-link the bookmark shape/color/dedup
  rules from the progress-sync field table
2026-08-20 14:40:59 -04:00
john-okeefe df90938c5c docs(api): document device sidecar endpoints; fix registration docs
- New get_sidecar_config.md for GET /api/devices/:id/sidecar and
  /sidecar/download: the .bookhoard.json config served to devices
  (endpoints, books keyed by per-format SHA-256 with UUID fallback,
  collections, format availability) used by the KOReader plugin to
  self-configure
- register_device.md: correct the response — no device_id at
  registration; auth_url is /devices/approve/:id (was the nonexistent
  /devices/auth/confirm/:id); document poll_interval and
  setup_instructions, and the approve-then-poll flow
- get_devices.md: fix the status endpoint path to
  POST /api/devices/register/status (was /api/devices/auth/status)
2026-08-20 14:40:54 -04:00
john-okeefe ffcdab36a0 docs(api): document hash-conflict resolution endpoints
Cover the admin API added in 03cb4c7 for duplicate-content decisions:

- GET /api/admin/hash-conflicts — pending conflict groups with member
  items and per-item usage counts (progress, highlights, bookmarks,
  notes, collections)
- POST /api/admin/hash-conflicts/:id/resolve — action=keep (merge child
  rows into keep_uuid, delete losers) vs action=keep_all (dismiss);
  JSON and form-encoded bodies, error codes including 409 for already
  resolved
- When conflicts are created (startup backfill, rescans) and the
  guarantee that files on disk are never deleted
2026-08-20 14:40:50 -04:00
john-okeefe 44a0f8c7a4 docs(api): document unified tunable system settings endpoints
The scattered scan-settings JSON routes are superseded by the new
admin-only /api/system/settings pair backed by the SettingsRegistry
(introduced in 885f6d8 / bc47450).

- Rewrite system/settings.md around GET/PUT /api/system/settings:
  SettingEntry metadata shape (type, min/max, requires_restart,
  category, group, is_default), type-aware validation rules, and the
  full tunable-setting catalog (scanner, general, security, api, sync,
  performance) with defaults, ranges, and restart requirements
- Note the legacy /api/libraries/scan-settings routes as back-compat
  only (they now refresh the registry cache on write)
- Add system/config.md for GET/PUT /api/system/config: raw key/value
  system configuration (e.g. base_url), including validation notes and
  guidance to prefer the typed settings endpoint for registry keys
2026-08-20 14:40:45 -04:00
john-okeefe 8ca95db08a docs(user): update admin and collections instructions for sidebar UI
Admin:
- Admin pages live in the sidebar's Administration panel (Dashboard,
  Libraries, Hash Conflicts, Users, Settings)
- Library creation is the Create Library modal (Library Name,
  Description, Library Type); folders are added afterwards by expanding
  the library row and using the Folders section's path input + Browse +
  Add — the old Add Library modal with folder and 'Scan on save' fields
  no longer exists
- Scanning is via the Scanner API or watch mode (File Watcher status on
  the admin dashboard); remove references to the removed per-library
  Rescan button and 'Force Rescan' option

Collections:
- Fill in the empty creating/managing placeholders with the real flow:
  New Collection button, modal fields (name, description, icon grid,
  color swatches), per-collection edit/delete icon buttons, Restore
  System button, and dashboard-section visibility via Customize
  Dashboard
2026-08-20 14:30:10 -04:00
john-okeefe 9a60196f1d docs(user): update dashboard and bookshelf instructions for sidebar UI
Dashboard:
- Customize Dashboard is opened from the icon button at the right end
  of the Library bar (next to Refresh), and requires a specific library
  selected rather than 'All Libraries'
- Library switching uses the Library dropdown in the bar below the top
  bar (includes 'All Libraries' with counts)
- Collection sections are shown/hidden from the Customize Dashboard
  toggles — the per-collection 'Show on Dashboard' setting is gone
- Mention the hover chevrons for scrolling carousels

Bookshelf:
- Saved filters: document the new toolbar buttons — Filters (opens the
  filter drawer with Apply Filters, Esc, and overlay-click close), Save,
  Load (Saved Filters dropdown with trash-icon delete), and Clear —
  replacing the emoji-labelled Save Filter / Saved Filters buttons
- Tag filtering: filters now live behind the Filters drawer on the All
  Books page
2026-08-20 14:30:05 -04:00
john-okeefe 7274b5196c docs(user): update appearance/profile instructions for sidebar UI
- Theme switching lives in the sidebar's Appearance panel (palette
  icon): swatch list with a checkmark on the active theme, and the
  'Bookshelf' section below it for wood textures; on small screens the
  sidebar opens via the top-bar menu button
- Profile: account menu is the username accordion at the bottom of the
  sidebar (not top-right); save button is 'Save Changes'
- Remove Wood Light/Dark/Mahogany from the Available Themes list (they
  are bookshelf backgrounds, not color themes) and consolidate the
  Catppuccin variants
2026-08-20 14:30:00 -04:00
john-okeefe 868003331c docs(user): update device and sync navigation for sidebar UI
The new-ui redesign replaced the top header with a sidebar and removed
the Settings pages.

- Replace all 'Settings → Devices' paths with the Devices page in the
  sidebar
- Conflicts are resolved from the book detail page's Sync Progress
  button or the Conflicts page (/conflicts); drop the nonexistent
  'Settings → Conflicts' path
- Queue status and unlinked-book references no longer invent per-device
  button paths that don't exist on the Devices page
- Reading history now points to the Progress page / book detail
- Export FAQ no longer references a Settings → Export flow that isn't
  in the UI
2026-08-20 14:29:54 -04:00
john-okeefe e310fa6a9d docs(index): update device guidance and fix broken links
- Device setup and quick-find entries now lead with the KOReader guide
  and label native Kobo sync as coming soon
- Kobo protocol/API listings tagged as a coming-soon feature
- Fix seven pre-existing broken links: contributing/Development.md had
  the wrong case (development.md), and PROJECT_GUIDELINES.md links were
  missing the ../ prefix to reach the repo root
- Refresh last-updated stamp
2026-08-20 14:14:55 -04:00
john-okeefe 2a7ac881fd docs(user): align sync and user guides with current device support
- sync-guide: only Web and KOReader are fully supported; move Kobo to
  coming soon, drop fake Q2-Q4 2026 release dates for mobile/Kindle/
  Remarkable, and describe the plugin + server-approval registration
  flow instead of QR-code/URL approval
- sync-guide: remove cellular/mobile-app advice from battery and
  best-practice sections, correct the Calibre compatibility FAQ, update
  the changelog to reflect shipped vs. pending sync features, and fix
  the license header (GPL-3.0, not MIT)
- user-guide: lead device setup with KOReader; mark the Kobo guide as
  coming soon
- calibre-integration: OPDS client list no longer implies native Kobo
  support
- auth overview: label the mobile-application token guidance as
  'coming later' since no mobile apps exist yet
2026-08-20 14:14:49 -04:00