Phase 1 (foundation): 'audiobooks' library type seeded (schema + DB);
audio_metadata JSONB on media_items (the kobo_metadata precedent —
only audio paths read it); reading_progress.audio_offset_seconds for
the honest {chapter, offset} locator; extensions/mimes registered for
ALL breadth formats (m4b/m4a/mp3/flac — deliberate per the DOC/LIT
doctrine, m4b scans richly first); classifyFormatGroup 'audiobook'
case (gotcha 39 recomputes on rescan); Go mime table extended at boot
so media elements accept the streams; ffmpeg/ffprobe in the image
(8.1.2 — also the xHE-AAC identification tripwire).
Phase 2 (ingest): sidecar-first (Libation/ABS metadata.json —
narrators/ASIN/curated chapters), ffprobe fallback/filler (duration,
bitrate, codec/profile, embedded chapters). Fixed three real
parser brittlenesses the Libation sidecar exposed: series accepts
string OR array-of-objects, publishedYear accepts number OR string,
ffprobe chapter timings accept number OR string (flexFloat). Folder-
structure fallback now runs on the UPDATE path too (rescans were
clearing sidecar-absent series/author); audio facts re-derive on
every rescan (gotcha 39).
Phase 3 (playback + sync): GET /api/media-items/:id/chapters (the
format-abstraction list); streaming rides the existing cookie-authed
Range /uploads path; PUT progress carries audio_offset_seconds +
elapsed_seconds (honest wall-clock playing time) — the reading-
history capture prefers player-reported elapsed over the inter-push
delta, same 2h ceiling; reading_mode='audio' marks the row.
Phase 4 (web player): dedicated audio reader page (foliate not
involved) — cover, chapter title, whole-book scrubber, skip ± (chips
1/5/10/15/30s, default 10s), speed (0.75×–2×), chapter list with
jump-to, Media Session lockscreen/notification transport, periodic
playing-only progress pushes, position restore on open.
Verified live: ingest of the Libation sample (Gardens of the Moon —
title/author/series/cover/date from sidecar+folders, 34 chapters,
26h duration, aac/LC); chapters endpoint; the reader route serving
the player; playback with position restore (0:57 round-trip);
progress rows in audio mode; wall-clock session rows in
reading_history.
Bug: the admin 'Scanning Libraries' widget fed the server's 0-1
progress straight into percent strings and bar widths, so the overall
and per-library percentages capped at '1%' and the bars never filled.
Both now scale by 100 (matching the setup page, which was already
correct).
Bug: scan state only reached clients via websocket events, so a page
opened (or refreshed) mid-scan showed nothing until the next tick —
and the admin widget never re-attached at all. The worker now exposes
active jobs (with their library_id so rows keep their names), served
as GET /api/scanner/active (admin, re-attaches the widget + polling)
and GET /api/scanner/active-summary (any user, restores the sidebar
indicator on load). Idle hides both, exactly as before.
Annoyance: the sidebar's 'Scanning… N%' was a block element between
the logo and the nav, shoving every menu entry down while a scan ran.
The percent now lives inside the fixed-height logo row next to the
spinner (spinner last, so growing digits extend leftward); nothing
below it moves.
The design parked in the handoff (multi-tier stats off reading_history;
bookorbit-style in data, not looks) is in. The real prerequisite work
was data: reading_history existed but NOTHING wrote to it — the whole
stats feature was sitting on an empty table.
- CAPTURE (no client changes, no repo-boundary crossings): every
successful progress write through SaveProgress (KOReader + app + web
+ Kobo + offline queue — the one chokepoint) now records one
reading_history session row: span = the interval since the PREVIOUS
write instant, pages = positive page delta when both sides know the
page index, marker rows for first contact. Spans over 2h are kept as
zero-second markers (overnight suspend→resume, offline flushes must
not inflate totals) — the honest 'sync cadence, not a stopwatch'
caveat stays. device_id only when a REAL device row matches (the
web/app progress path historically stuffs the USER id into DeviceID
— probe instead of trusting it; FK violation would silently drop
every capture). Source preserved in device_metadata.
- API: /api/stats/summary?range=7|30|90|year (totals, zero-filled
daily arrays, streaks, most-active day/weekday, top books, library
breakdown — one call), /api/stats/this-week (widget: rolling 7d vs
previous + streak), /api/stats/books/:id (lifetime + recent
sessions). All bucketed in the user's timezone; finished ≡ any push
≥0.995; visible-library lens enforced at source. Legacy
/api/analytics/* kept untouched. Range windows anchor end-of-day to
the START of today (blasting the clock time into tomorrow resolved
'today' a calendar day late — found by the streak test).
- /stats page (sidebar Stats between Progress and Devices, previously
/analytics was an orphan): range selector, five stat cards, two
Chart.js v4 charts (daily minutes bar, pages/day line — chunked via
dynamic import so main.js stays untouched), per-library breakdown,
most-read list, empty-state explainer. /analytics now 301s to /stats.
The orphaned analytics.templ + analytics.ts removed (their containers
and the page JSON never matched; the page was astub).
- Dashboard 'This Week' widget (server-rendered, suppressed on service
failure) and the book detail 'Your Reading Stats' card (lifetime
totals of the viewer's own sessions) complete the tiers: per-user
default, per-library breakdown inline, per-book on the page,
per-system/admin aggregate parked as before.
Tests: statistics_test.go (capture semantics at the chokepoint,
tz/day buckets, zero-fill, streak math, finished detection, range
validation, auth gate, widget, per-book lifetime). Full suite PASS.
The middleware's per-request user lookup (ghost-session guard) fetched
the full row and discarded it, then built the context stub without a
timezone — so every server-side conversion via the context user fell
back to UTC no matter what the handlers did. The stub now carries
Timezone from that same lookup (fetchLiveUser).
Live-verified: progress page renders 02:03 PM for a 14:03 New York
instant; an analytics session at 02:00 UTC buckets to 10-12 (10pm NY
the previous day) instead of 10-13.
The static handler sent no Cache-Control, so browsers fell back to
heuristic caching (10% of age since Last-Modified) and kept serving a
stale reader.js for hours after a deploy — fixed web bugs reported as
"not fixed". no-cache forces revalidation every load; the filesystem
Last-Modified/If-Modified-Since pair keeps unchanged loads cheap.
Found while closing out the PDF adjust v2 session: the user's Chrome
was running a pre-endpoint bundle, which read as "the web has not
been fixed".
The guard returned c.JSON(401, ...), whose nil error told echo the
SuccessHandler succeeded — the pipeline then continued into the route
handler, which type-asserted the never-set "user" context value and
panic-recovered as a 500 appended after the 401 body. The status code
happened to survive (first write wins), which is why the pin test
passed, but every ghost request also executed the target handler with
no user — the exact unauthenticated-execution class the guard exists to
prevent.
Return echo.NewHTTPError(401, ...) instead: non-nil aborts the chain
and renders {"message":"user_not_found: ..."}; the pin test now asserts
that shape. Full suite: 219 groups, zero failures.
The JWT middleware validated signature + expiry and fabricated the
request user purely from token claims, so a signature-valid token for a
DELETED user kept full API access until token expiry — observed live on
hardware (a phone session survived the test suite's user-table
recreation and kept serving its true empty sets; open handoff bug 1).
An ensureUserExistsMiddleware already existed but was only wired on the
frontend SSR group (frontend.go) — every /api group used bare
createJWTMiddleware. The existence check now lives INSIDE
createJWTMiddleware's SuccessHandler so every protected group (SSR and
API alike) gets it:
- After claim validation, GetUser by the token's user_id; a missing row
clears the token cookie and answers 401 user_not_found for API calls
(the app's TokenAuthenticator then re-logins with stored credentials)
or redirects to /login?session=invalid for page requests — same
shape-aware split the error handler already uses.
- Cost: one primary-key lookup per authenticated request (~0.2ms local);
instant revocation, no TTL semantics. The redundant SSR-only
middleware is deleted (its redirect behavior is preserved inside the
guard).
The guard exposed two processing-issues tests that were unknowingly
RELYING on ghost semantics: they nested a second setupTestServer whose
bootstrap deleted the first setup's regular user, then asserted 403
from the dead token. De-nested: a createLibraryDirect helper inserts
the library under the live setup's user, no second bootstrap.
New TestDeletedUserTokenRejected pins the contract: token works while
the user exists -> 401 user_not_found after deletion. Full suite: 219
groups, zero failures.
Complete the SHA-256 lifecycle for preexisting databases: items
imported before hashing existed get hashed automatically, and any
content duplicates discovered in the process land on the new admin
Hash Conflicts page for an explicit keep/merge decision.
HashBackfillService (runs once 30s after startup, independent of
auto-scan):
- hashes every media_items row where file_sha256 IS NULL, resolving
each path through LibraryService; per-item failures are logged and
skipped so one unreadable file cannot block the pass
- no-op once everything is hashed (logged and skipped)
- finishes with a conflict sweep flagging every content-duplicate
group via FindHashConflictGroups + CreateHashConflict; the sweep
runs after the per-item pass because a preexisting pair only
becomes detectable once both sides have their hash
API (admin-only):
- GET /api/admin/hash-conflicts - pending groups with member items
and usage counts
- POST /api/admin/hash-conflicts/:id/resolve - action=keep_all, or
action=keep with keep_uuid: validates the uuid belongs to the
group, re-parents every other copy's child rows onto the kept item
(reparent_media_item_children), deletes the losers, and records
the resolution + resolving admin; accepts form or JSON bodies and
returns the htmx resolved fragment
Page route /admin/hash-conflicts (admin-only) renders the template
with hydrated conflict data; HashConflictsHandler wired into the
router Config and constructed in main.
Verified end-to-end against the live database: duplicate detection,
pending listing, keep_all resolution, merge path (re-parent +
delete), and - critically - a resolved group is not re-flagged by a
later sweep (upsert no-op). Database restored afterward.
Construct the SettingsRegistry at boot, load it, and thread it through
every consumer so the configurable values take effect and stay cached.
cmd/server/main.go:
- Build the registry from the Queries handle and Load() it right after
schema init; a load failure logs and continues (getters fall back to
compiled defaults, so startup is never blocked).
- Wire the registry into the package-level password validator
(SetDefaultPasswordSettings) and call SetSettings on every handler/
service that reads tunables: AuthHandler, DeviceAuthMiddleware,
OPDSHandler, SidecarHandler, SystemSettingsHandler,
AnnotationService, ConversionService.
- Source the restart-time values from the registry: login lockout
(max attempts + duration) feeds NewLoginAttemptTracker, and the new
NewSyncQueueProcessorWithConfig / NewWorkerWithConfig take the sync
queue and worker pool configs.
router.go:
- Config gains a Settings *database.SettingsRegistry field.
- The global auth rate limiter now reads RequestsPerMinute from
registry.AuthRateLimit() (env stays as the enabled/disabled switch
and as the fallback if the registry is unset).
admin_library.go:
- The HTMX scan-settings save endpoint reloads the registry after
writing so the change is visible without a page reload.
- Add PUT /admin/settings/tunable: a small HTMX endpoint that calls
SystemSettingsHandler.ApplySetting and returns a colored status
snippet ("Saved" or "Saved — restart required") for the admin UI's
per-row forms.
Three bugs fixed:
1. Schema seeded base_url with fake placeholder 'bookhoard.example.com'.
Removed seed; startup now seeds from BASE_URL env var only if DB row
is empty (admin changes persist across restarts). One-time UPDATE
clears the placeholder in existing installs.
2. config.GetBaseURL() had a broken type assertion (local SystemConfigRow
vs database.SystemConfig) that always failed, returning . Admin panel
showed env var fallback instead of actual DB value. Fixed with a
function-type getter that properly wraps the DB query.
3. OPDS handler read base_url only from DB with no fallback. When DB had
the placeholder, all feed links pointed to an unreachable domain,
breaking KOReader search/download. Added deriveBaseURL() helper that
falls back to the request Host/scheme when DB value is empty.
Setup gate improvements:
- isSetupComplete now requires both admin user AND non-empty base_url
- Setup middleware no longer exempts all /api/ routes; only allows
/api/auth/register, /api/auth/login, /api/system/config before setup
is complete. All other API routes get 503.
- Cache invalidated when base_url is saved via admin settings
Dev workflow:
- New bruno/NewDevDBSetup/SetBaseUrl.yml for dev DB setup
- NewDB.sh runs SetBaseUrl between RegisterUser and CreateEbookLibrary
Complete the annotation sync pipeline across all ingest and serve paths.
Previously, annotations sent inline with KOReader progress pushes were
silently discarded, and no annotations were ever served back to devices.
INGEST (device → server):
KOReader (koreader.go):
- Add processBookAnnotations helper that processes inline highlights,
notes, and bookmarks from every progress push (immediate + checkpoint)
- Highlights get CRE→CFI position conversion before SaveHighlight
- KOReader 'notes' (text + notes) stored as highlights with NoteText
to ensure correct round-trip classification
- Bookmarks routed through SaveBookmark with device sync data
- Called from both updateProgressForBook and handleCheckpointSync
Kobo (kobo.go):
- Markup handler: annotations and bookmarks route through
AnnotationService (SaveHighlight/SaveBookmark)
- Bookmark handler: same routing with device sync data
- SyncFromServer handler: same routing
- All handlers fall back to direct DB calls when annotationSvc == nil
Web reader (media.go):
- CreateMediaHighlight → SaveHighlight (Source="web", ModifiedAt=now)
- CreateMediaNote → SaveNote (Source="web")
- DeleteMediaHighlight → TombstoneHighlightByID
- DeleteMediaNote → TombstoneNoteByID (was hard delete, now tombstone)
- All fall back to old behavior when annotationSvc == nil
SERVE (server → device):
KOReader GetMetadata (koreader.go):
- Query and serve bookmarks from media_bookmarks table (was missing)
- Serve deleted_highlights and deleted_bookmarks arrays containing
device_sync_data + dedup_key for client-side deletion
- Highlights/notes already served with reverse CFI conversion
Kobo Markup handler (kobo.go):
- Track processed books during sync
- Query tombstones per book, extract bookmark_id from device_sync_data
- Return DeletedAnnotations array in KoboSyncStatus response
Conflict resolution (conflicts.go):
- Enable annotation conflict types in ResolveConflict handler
- Add applyAnnotationResolution dispatching to:
applyHighlightResolution / applyBookmarkResolution / applyNoteResolution
- Each looks up by dedup_key and applies winner's fields
- Allow manual override of auto_resolved conflicts
(changed check from != "unresolved" to == "user_resolved")
Infrastructure:
- AnnotationService field + SetAnnotationService in router Config
- Inject AnnotationService into KOReader, Kobo, Media handlers
- Start tombstone purger goroutine in main.go (24h interval)
- Test helpers: construct AnnotationService in test setup
When a Kobo device pushes a last-read-place bookmark, the server now
converts the KEPUB CFI (with koboSpan wrappers) to a standard EPUB CFI
and extracts surrounding text as context_text for use by other devices
(KOReader, web reader) during their pull-side CFI conversions.
Previously the raw KEPUB CFI was stored verbatim as epubcfi, which
meant foliate and CREngine couldn't resolve it (wrong child indices
due to koboSpan wrappers), and no context_text was available for the
text-search fallback in ConvertStandardToCRE.
Changes:
- kepub_cfi_converter.go: Add ExtractedContext field to
KEPUBConversionResult, populated from the already-computed
searchText in both ConvertKEPUBCFIToStandard and
ConvertStandardCFIToKEPUB (exact-match and percentage-fallback
paths).
- kobo.go: Add libraryService field and SetLibraryService setter
(mirrors KOReaderHandler pattern). Add convertKoboCFIToStandard
helper that resolves EPUB+KEPUB paths, instantiates the converter,
and returns the converted CFI + extracted context. The last-read-place
branch in Markup now calls this helper for reflowable formats,
skipping fixed-layout/comic archives (page-index only).
- router.go: Add LibraryService to router Config.
- sync.go: Wire LibraryService to KoboHandler via SetLibraryService.
- main.go: Pass libraryService through router config.
The conversion is purely additive — if no KEPUB file exists on disk
(e.g. side-loaded EPUB without kepubify conversion), the handler
gracefully skips conversion and stores the raw CFI as before.
Add setupRedirectMiddleware that checks the setup_complete system
setting on every request. If setup is incomplete, all non-setup
requests are redirected to /setup so the wizard is the first thing
new users see. The check uses an in-memory cache (10s TTL) to avoid
hitting the database on every request, with cache invalidation on
setup completion.
The middleware skips /setup, /api/*, /static/*, /health, and
/favicon.ico so the wizard page, API calls, and static assets load
normally during setup.
Register two new routes:
- GET /setup: renders the setup wizard SSR template
- PUT /api/setup/complete: marks setup as complete (JWT-protected,
requires an authenticated admin user created in step 1)
Create SeriesHandler with two API endpoints:
- GET /api/series (paginated series list with covers)
- GET /api/series/books (books in a specific series)
Uses query param ?name=X instead of path param to avoid URL encoding
issues with special characters in series names.
Add GetSeriesCardsData helper returning services.SeriesInfo for use
by the SSR route (avoids handlers→templates import cycle).
Register /api/series routes via registerSeriesRoutes in router.
Add /series SSR route in frontend.go with library-scoped pagination
and error handling, matching the dashboard/bookshelf patterns.
Add SeriesHandler to router Config and instantiate in main.go.
Add SeriesCardData type to templates/types.go.
Add Continue Series as the 5th valid system collection in
dashboard handler and auth handler's CreateDefaultCollectionsForUser.
- Remove GET /progress/:id and POST /progress/:id from progress routes.
These were superseded by the media-item progress routes. Only
GET /progress/:id/history remains.
- Add ProgressService to router.Config so sync.go can inject it into
KoboHandler via SetProgressService().
- Inject ProgressService into KoboHandler at route registration time
rather than requiring a separate setup step.
- Update comment from 'Legacy progress routes' to 'Progress routes'.
Remove redundant type conversions that Go 1.26 makes unnecessary or that
were already no-ops:
- uuid.UUID(x.Bytes) → x.Bytes (uuid.UUID is [16]byte, same as pgtype UUID Bytes)
- pgtype.UUID{Bytes: [16]byte(u), Valid: true} → pgtype.UUID{Bytes: u, Valid: true}
- (*time.Time)(&x.Time) → &x.Time
- json.RawMessage(x) → x where x is already []byte
- []byte(stringVal) → stringVal where []byte is expected
- int()/int64()/byte() casts on values already of the target type
- Decompressor(fn) → fn (type is identical)
Handle previously ignored error returns:
- collections.go: check json.Unmarshal error in GetCollection
- conversion_service.go: check fileSize.Scan() error
- app_test.go: check app.Shutdown() error in benchmark
Wire up the ProcessingIssuesHandler throughout the application:
cmd/server/main.go:
- Remove obsolete commented-out getTemplateUserWithTheme function
- Instantiate ProcessingIssuesHandler with database queries
- Add handler to router Config (with field alignment cleanup)
internal/router/router.go:
- Add ProcessingIssuesHandler field to router Config struct
- Reformat Config struct for better field alignment
This enables the processing issues API endpoints for listing and getting
statistics about issues within libraries, integrated with the admin UI.
- Add modular dockable panel architecture with:
- Panel dock system (drag, lock, snap-back, window-shade)
- TOC, Settings, Navigator, Bookmarks as dockable components
- Lock toggle to prevent accidental moves
- Snap-back to last valid position if dropped in invalid area
- Per-user layout stored in reader_settings JSONB
- Update TypeScript types with PanelLayoutSettings and PanelState
- Add panel-dock-system.ts implementation to plan
- Add navigator-panel.ts for Affinity-style page navigation
- Update reader template with dockable panels and lock buttons
- Add Section 2.4 with database/Go implementation issues and fixes
- Document schema changes (DECIMAL -> REAL)
- Document all type fixes needed in reader.go
Add complete backend implementation for saved filters CRUD operations
with proper service layer architecture and RESTful API endpoints.
Service Layer (internal/services/filters.go):
- NewFiltersService() constructor following project patterns
- GetSavedFilters(): Retrieve all filters for user + resource type
- CreateSavedFilter(): Create filter with duplicate name validation
- UpdateSavedFilter(): Update filter with ownership verification
- DeleteSavedFilter(): Delete filter with user scoping
Business Logic:
- Filter name uniqueness enforced per user + resource type
- User ownership validation on all operations (JWT user_id)
- JSONB marshaling/unmarshaling for flexible filter storage
- Proper error wrapping with context messages
Handler Layer (internal/handlers/filters.go):
- NewFiltersHandler() constructor (receives db.Queries)
- GetSavedFilters: GET /api/saved-filters?resource_type=X
- CreateSavedFilter: POST /api/saved-filters
- UpdateSavedFilter: PUT /api/saved-filters/:id
- DeleteSavedFilter: DELETE /api/saved-filters/:id
Content Negotiation:
- Supports both JSON (API clients) and HTML (HTMX) responses
- wantsHTML() helper checks Accept header
- HX-Redirect header for HTMX form submissions
- Proper status codes (200, 201, 204, 400, 401, 404, 409)
Router Configuration:
- registerFiltersRoutes() function in internal/router/filters.go
- JWT middleware protection on all endpoints
- RESTful route structure: /api/saved-filters
- Registered in main router.go RegisterRoutes() function
- Added FiltersHandler to router.Config struct
Test Infrastructure:
- Added FiltersHandler to test server setup (test_helpers_test.go)
- FiltersHandler initialized in setupTestServer() function
- Router.Config includes FiltersHandler for integration tests
Code Quality:
- Follows PROJECT_GUIDELINES.md service layer patterns
- Uses database models (not custom domain models)
- JSONB returned as []byte (matches collections pattern)
- All errors wrapped with context using fmt.Errorf
- Handlers create services internally (not dependency injection)
Part of: Saved Filters Implementation (Phase 2: Backend)
Related: #saved-filters-feature
Phase 1: Register routes that were defined but never connected
- Add GET/PUT /api/system/config routes (admin-only) for system
configuration in new internal/router/system.go
- Add GET /api/devices/:id/sidecar routes for device sidecar config
- Add SidecarHandler to router Config struct
- Instantiate SidecarHandler in main.go with config for fallback support
These routes were implemented in handlers/sidecar.go but never registered,
breaking the ability to configure base URLs for device sync.
Clean up internal/router/router.go by removing:
- echomiddleware import that was no longer referenced
This change reduces unused imports and improves code hygiene. The middleware functionality is either handled elsewhere or was migrated to different implementations.
Update all router files to use Echo v5 APIs and type signatures.
Changes in router.go:
- Replace echomiddleware.Logger() with RequestLogger() (line 144)
- Update import from echo/v4 to echo/v5
Changes in frontend.go:
- Update frontend handler signatures to use *echo.Context
- Fix middleware registration for v5 compatibility
Changes in auth.go, library.go, scanner.go, sync.go, helpers.go:
- Update handler function signatures to *echo.Context
- Ensure consistent type usage across all route handlers
All routes now properly implement Echo v5's middleware and handler patterns.
- Fix SearchMediaItems to retrieve user object from context instead of string
- Remove redundant UUID parsing, use user.ID directly
- Add error logging for search failures with query details
- Fix JWT middleware to use echo.NewHTTPError for consistent error format
- Improves debugging and error response consistency across API
Extract health check logic into GetHealth method on Config struct and
integrate with Worker service for accurate scan status reporting.
Changes:
- Move health check handler from inline function to Config.GetHealth()
- Add Worker field to Config struct for dependency injection
- Wire Worker into main server dependencies
- Report actual scan_in_progress status using Worker.HasActiveScans()
- Report actual active_jobs count using Worker.GetActiveJobCount()
This provides more accurate health monitoring by checking the real state
of background jobs rather than returning static placeholder values.
- Add JobsHandler with CreateJob and GetJobStatus endpoints
- Add jobs router with POST /api/jobs and GET /api/jobs/:jobId routes
- Integrate JobsHandler into main server and router config
- Add check for auto_scan_enabled setting in router before starting watch mode
- Update StartScanner handler to verify auto-scan is enabled
- Switch scanner to use watchModeCtx/watchModeCancel instead of ctx/cancel
- Update StartWatchModeForLibrary to use new MediaScanner signature
- Implement event queue with 3-second debouncing for file system events
- Add configurable polling fallback (default 3 min) via SCAN_POLL_INTERVAL_MINUTES
- Add SyncFilesystemWithDatabase to detect orphaned DB entries and new files
- Integrate utils.ResolveMediaURL for consistent media file path resolution
- Add COOKIE_SECURE env var with SameSite=LaxMode for session cookies
- Update media handler to properly decode URL paths for file serving
- Refactor scanner initialization to accept poll interval configuration
Fixed a critical bug where watch mode failed to start automatically during container
initialization, despite comments in app.go:79 claiming it would start "after 2-second delay."
Root Cause:
- StartWatchModeForAllLibraries() function existed but was never invoked
- Comment in app.go claimed watch mode started automatically, but no startup code existed
Changes:
- Added "context" import to internal/router/router.go
- Added goroutine in configureRouter() that:
* Waits 2 seconds after server initialization
* Calls StartWatchModeForAllLibraries() to activate monitoring
* Logs startup status or errors
This ensures watch mode begins scanning for new media files automatically when the
container starts, rather than requiring manual intervention.
Testing: Verified watch mode now activates automatically in container logs.
The TokenLookup config was missing the Bearer prefix stripper, causing
all authenticated requests to fail with 'token is malformed'. The JWT
library was trying to decode 'Bearer eyJh...' as a token, failing at
the space character.
Changed from: 'cookie:token,header:Authorization'
Changed to: 'cookie:token,header:Authorization:Bearer '
This fixes all integration tests that use Bearer token authentication.
- Add renderErrorPage helper for consistent error rendering
- Add ensureUserExistsMiddleware to detect deleted users and redirect to login
- Add catch-all 404 handler for unknown routes
- Gracefully handle data loading failures with error messages instead of crashing
- Log errors for debugging while still rendering pages
Add DashboardService and DashboardHandler to application configuration:
Router Config Updates (internal/router/router.go):
- Add services import for DashboardService type
- Add DashboardService field to Config struct
- DashboardService: Used by SSR routes in frontend.go for data fetching
- DashboardHandler: Used by API routes in dashboard.go for JSON endpoints
Server Initialization (cmd/server/main.go):
- Create dashboardService instance using services.NewDashboardService(queries)
- Keep dashboardHandler creation (already exists from Phase 4)
- Add DashboardService to routerConfig
- Both services now available for dependency injection
Test Helpers (cmd/server/tests/test_helpers.go):
- Create dashboardService instance for testing
- Create dashboardHandler instance for testing
- Add both DashboardService and DashboardHandler to routerConfig
- Ensures test environment matches production setup
Architecture Rationale:
- DashboardService: Service layer with business logic (reusable by SSR, mobile)
- DashboardHandler: HTTP handler layer (JSON API endpoints)
- Separation allows SSR templates to call service directly
- API routes use handler for proper HTTP response handling
- Mobile apps can use API endpoints via DashboardHandler
All three files updated consistently for complete integration.
Add dashboard route registration and wire up handler:
Router Changes:
- Add DashboardHandler to router.Config struct
- Create internal/router/dashboard.go with dashboard route registration
- Register dashboard routes in main RegisterRoutes function
Dashboard Routes (all protected by JWT):
- GET /api/dashboard/sections: Get dashboard sections for user
* Query params: library_id (required), limit (optional, default 20, max 100)
* Returns: JSON with sections array
- PUT /api/dashboard/preferences: Update dashboard preferences
* Body: library_id, hidden_collections, collection_order, items_per_section
* Returns: Updated preferences
- POST /api/dashboard/restore-system-collection: Restore system collection to defaults
* Body: collection_name (must be valid system collection)
* Returns: Success message
Server Integration:
- Create dashboardHandler in cmd/server/main.go
- Add dashboardHandler to routerConfig
- Routes are automatically registered on server startup
- Add strings import for Accept header parsing
- Add wantsHTML() helper function to detect HTML vs API requests
- Checks Accept header for text/html
- Checks HX-Request header for HTMX requests
- Checks X-Requested-With for AJAX (should return JSON)
- Defaults to JSON for API routes
- Update JWT middleware ErrorHandler to:
- Redirect HTML requests to /login?session=expired
- Return JSON error for API requests with session_expired message
- Enables browser navigation to redirect gracefully while API calls
return proper error responses
This fixes the issue where protected routes returned JSON 401
for browser navigation instead of redirecting to login.
- Add ScannerHandler field to Config struct for frontend route access
- Move scannerHandler creation before registerFrontendRoutes call
- Enables /progress page to access scanner data
- Serve static files from web/static directory
- Add theme class safelist to Tailwind config for dynamic theming support
- Regenerate CSS with updated configuration
- Add LibraryTestData struct to TestDeviceSetup
- Implement CreateLibrary() for proper library creation in tests
- Implement CreateCollection() for test collection support
- Improve test isolation with dedicated library creation
This provides a more robust foundation for integration tests that need
proper library management support.
Critical fixes to prevent goroutine leaks during application shutdown:
1. Sync Queue Processor:
- Changed StartCleanupTask() to return context.CancelFunc
- Modified to accept and watch cancellable context
- Added queue context/cancel to Handler struct
- Created StartBackgroundTasks() method for main handler instance
- Cancel queue processor during shutdown in StopScheduler()
2. Connection Manager:
- Modified StartCleanupTask() to use cancellable context
- Returns cancel function that can be called during shutdown
- Goroutine now properly exits when context is cancelled
3. Handler Lifecycle:
- Added StartBackgroundTasks() to Handler
- Only main handler instance starts background goroutines
- Temporary handler instances (library/sync routes) don't start tasks
- StopScheduler() now properly shuts down all background goroutines
4. Router Integration:
- Updated SetupRoutes to accept queueProcessor parameter
- Main scanner handler starts background tasks after creation
- Library and sync route handlers don't start duplicate tasks
Impact:
- Fixes 2 major goroutine leaks (queue processor + connection cleanup)
- Application now properly shuts down all goroutines on exit
- No more resource leaks from long-running goroutines
- Test added to detect future goroutine regressions
Test: TestGoroutineCleanup verifies background services can be stopped.
- Update comment: 'ebook handler' -> 'scanner handler'
- Rename ebookHandler variable to scannerHandler in router.go
- Update registerProgressRoutes parameter name
- Update registerScannerRoutes calls to use new variable name
- Update Config struct to replace EbookHandler with MediaHandler, SearchHandler, MatchingHandler
- Add CollectionHandler to Config struct
- Update RegisterRoutes to call new registration functions
- Create registerCollectionsRoutes for 15 collection endpoints
- Create registerSearchRoutes for search endpoints (uses MediaHandler.SearchMediaItems, MatchingHandler.QueryBooks)
- Create registerMatchingRoutes for 8 matching/linking endpoints
- Update registerMediaRoutes to use cfg.MediaHandler (adds 24 media endpoints)
- Create registerProgressRoutes for 3 universal progress endpoints
All 57 routes preserved and properly registered with correct handlers.
No functionality lost, all endpoints work identically.
This is Phase 5 of the ebook.go refactoring plan.
Remove the CollectionHandler field from router.Config struct and its
initialization in main.go. This field was never used - collections are
registered directly in handlers.SetupRoutes() where a CollectionHandler
is created locally.
Changes:
- Remove CollectionHandler field from internal/router/router.go Config
- Remove CollectionHandler: nil line from cmd/server/main.go
This cleans up dead code from the router refactoring. Collections
continue to work correctly as they are registered in SetupRoutes().
Related: Router refactoring completion
Phase 1 of scanner restoration plan
Changes:
- cmd/server/main.go: Capture ebookHandler from router.RegisterRoutes
- cmd/server/main.go: Start scheduler in background goroutine
- cmd/server/main.go: Defer StopScheduler() for graceful shutdown
- cmd/server/main.go: Start watch mode for all libraries after 2-second delay
- internal/router/router.go: Return ebookHandler from RegisterRoutes
This restores critical functionality that was removed during router refactor:
- Auto-scanning now works again
- Watch mode starts automatically for all libraries
- Graceful shutdown properly stops scheduler
Fixes issue where scheduler and watch mode were not starting on server boot.
Add createJWTMiddleware helper that sets database.Users object in context,
matching the original main.go JWT middleware behavior. This fixes
'authentication context error' panics in handlers that call
MustGetAuthenticatedUser.
Changes:
- Add createJWTMiddleware() in router.go
- Update all route files to use the helper
- Set user claims AND database.Users object in context
Add stub implementations for:
- library.go: Library management routes (admin + user visibility)
- device.go: Device registration and management routes
- router.go: Updated to import jwt package
Router package structure is complete with all route groups defined.
Next step: Incrementally migrate routes from main.go by calling
router.RegisterRoutes() and removing duplicate definitions.
All verification checks pass (26/26).
Create internal/router/ package to organize route registration:
- router.go: Main router setup and configuration
- auth.go: Authentication routes (login, register, profile, etc.)
- docs.go: Documentation routes
- frontend.go: Frontend SSR routes (/, /login, /admin, etc.)
- helpers.go: Helper functions for template rendering
This is the first step in refactoring 858-line main.go into
a more maintainable structure following Go best practices.
Routes themselves have NOT changed - only organization.
John O'Keefe
john-okeefe