The media scanner never populated page_count or total_characters in
media_items, leaving progress display and reading position calculations
with no reliable data. This commit fixes data population for all formats:
Comics (CBZ/CBR/CB7/CBT):
- Add countArchiveImages() helper that walks archive entries and counts
image files (.jpg, .jpeg, .png, .gif, .webp)
- Call it during comic metadata merge to set metadata.PageCount
PDFs:
- Extract pdfInfo.PageCount from the pdfcpu library (already available
from PDFInfo call, just never used) and set metadata.PageCount
Reflowable EPUBs:
- Use book.AllChaptersText() to compute metadata.TotalCharacters
- Use book.ChapterCount() to set metadata.ChapterCount
Fixed-layout EPUBs (manga/comics in EPUB format):
- Merge .epub into the .cbz case in countArchiveImages since both are
ZIP archives with images
- Detect fixed-layout EPUBs via DetectFixedLayoutEPUB() in both the
Calibre sidecar path (mergeMetadata) and the no-sidecar path
(extractMetadata), counting images when fixed-layout is detected
Format group on creation:
- Remove the guard condition on UpdateMediaItemFormatGroup so that
format_group, is_reflowable, and has_fixed_layout are set immediately
for every new item (not just items with text data)
- Use DetectFixedLayoutEPUB() instead of hardcoding all .epub as
reflowable, correctly classifying fixed-layout EPUBs
Also pass PageCount to CreateMediaItem and add PageCount,
TotalCharacters, and ChapterCount fields to the MediaMetadata struct.
Go's new() builtin takes a type and allocates a zero value — it cannot
wrap an expression. All instances of new(someExpression) were compile
errors. Replace each with a local variable assignment and address-of
operator.
Affected files:
- handlers/koreader.go: progress field pointers (Chapter, Page, etc.)
- handlers/kobo.go: pagesRemaining pointer
- handlers/queue.go: uuidPtrToString and timestamptzPtrToString helpers
- router/reader.go: bookmark pageNumber and chapterNumber pointers
- services/media_scanner.go: validation error message pointers
- services/worker.go: StartedAt and CompletedAt timestamps
- sync/offline.go: GetDeviceStatus return pointer
- tests/device_test.go: SyncEnabled and SyncFrequencyMinutes pointers
The reader failed to load comics and manga (and any file with special
characters in its path) for two reasons:
1. FileURL was built with raw fmt.Sprintf instead of ResolveMediaURL,
so characters like '#' in paths (e.g. 'Annual #2') were interpreted
as URL fragments, truncating the path and causing 404s.
2. The Alpine x-init expression used raw string interpolation for config
values, so apostrophes in paths (e.g. "I'll Use My Appraisal Skill")
broke JavaScript parsing with 'Unexpected identifier'.
Fix by using utils.ResolveMediaURL for proper URL path encoding and
json.Marshal for the initReader config to safely escape all special
characters.
Cover image URLs with special characters like parentheses, #, ?, or
spaces would break because browsers interpret them as URL delimiters.
Apply url.PathEscape() per path segment in ResolveMediaURL so the
server can correctly resolve files like "Wonder Woman (2016) #001.cbz.cover.jpg".
Also adds a package doc comment and fixes the exported function comment.
Comic archive formats (.cbz, .cbr, .cb7, .cbt) and .kepub files were
falling through to the default case in extractMetadata(), which only
set the title from the filename. This meant ComicInfo.xml was never
parsed and no cover images were extracted for comics without a Calibre
metadata.opf sidecar file.
The fix adds dedicated switch cases:
- .cbz/.cbr/.cb7/.cbt: calls mergeMetadata() with nil, which triggers
existing ComicInfo.xml parsing (title, series, issue number, writer,
publisher, genre, reading direction, etc.) and cover image extraction
from the archive. Falls back to sidecar cover if no image is found.
- .kepub: treated the same as .epub since KEPUB is an EPUB variant,
enabling full metadata and cover extraction.
Both ListProcessingIssues and GetProcessingIssueStats were reading the
URL parameter 'libraryId', but the routes in internal/router/library.go
define the param as ':id'. This caused both endpoints to always fail with
an invalid library ID error since c.Param('libraryId') returns an empty
string that can't be parsed as a UUID.
Remove redundant type conversions that Go 1.26 makes unnecessary or that
were already no-ops:
- uuid.UUID(x.Bytes) → x.Bytes (uuid.UUID is [16]byte, same as pgtype UUID Bytes)
- pgtype.UUID{Bytes: [16]byte(u), Valid: true} → pgtype.UUID{Bytes: u, Valid: true}
- (*time.Time)(&x.Time) → &x.Time
- json.RawMessage(x) → x where x is already []byte
- []byte(stringVal) → stringVal where []byte is expected
- int()/int64()/byte() casts on values already of the target type
- Decompressor(fn) → fn (type is identical)
Handle previously ignored error returns:
- collections.go: check json.Unmarshal error in GetCollection
- conversion_service.go: check fileSize.Scan() error
- app_test.go: check app.Shutdown() error in benchmark
DismissAllResolved was calling ListSyncConflictsByUser which filters to
'unresolved' conflicts only, so it could never find the user_resolved or
bulk_resolved conflicts it was trying to delete. The query always returned
an empty set, making dismiss-all a no-op.
Fix the leading space in three SQL query name annotations (ListConflictsByUser,
ListAllConflictsByUserAndStatus, CheckForProgressConflicts) that prevented
sqlc from generating their Go functions. Regenerate the query code and swap
DismissAllResolved to use ListConflictsByUser (no status filter) — the
existing Go loop already filters by resolution_status.
Replace direct error equality check with errors.Is() in media_scanner_hash_test.
In analytics_test.go, improve defer patterns by capturing resp.Body as a named parameter
to avoid stale references, and add require.NoError() checks on all json.NewDecoder().Decode()
calls that were previously silently ignoring decode errors.
Replace direct error equality checks (err == pgx.ErrNoRows, err != http.ErrServerClosed)
with the idiomatic errors.Is() function throughout handlers, services, middleware, and app
startup. This correctly handles wrapped error chains.
Also replace a raw type assertion (*HTTPError) with errors.AsType[*HTTPError]() in the
error handler middleware for consistency.
Additionally, rename shadowed variables for clarity:
- sidecar.go: config -> sidecarConfig, systemConfig (shadowed package-level vars)
- media_scanner.go: uuid -> uuidString (shadowed the uuid package import)
Two changes to EPUB metadata extraction:
1. Restructure extractMetadata so that fixed-layout detection and cover
extraction always run for EPUBs, even when extractEPUBMetadata returns
an error. Previously, a partial failure from the EPUB parser would skip
cover and format detection entirely, leaving books without covers.
2. Remove the language restriction (ja/jpn) from manga reading direction
detection. Manga tagged with 'manga' should default to RTL regardless
of the language metadata, since the tag is an explicit signal from the
user or metadata source.
In applyProgressResolution and applyResolution, currentPage and totalPages
were unconditionally read from existingProgress even when the preceding
query returned err (no rows). This caused a nil pointer dereference when
no existing reading progress existed for a media item. Now declare the
variables as zero-value pgtype.Int4 and only populate them from
existingProgress when err is nil.
Replace direct equality checks (err != pgx.ErrNoRows) with the idiomatic
errors.Is(err, pgx.ErrNoRows) pattern. This is the recommended Go practice
for error comparison as it correctly handles wrapped errors from error
chains, making the code more robust against future refactoring that might
wrap errors with fmt.Errorf and %w.
When opening a sevenzip archive, the init() method calls sr :=SevenZipReader()
but never checked if sr was nil before using it. This could cause a nil
pointer dereference when processing malformed or empty archives. Add an
explicit nil check returning errFormat early if the subreader is nil.
Also fixes a minor import grouping whitespace issue.
Add table-driven tests for the conflict handler's source selection
methods: GetMostRecentSource, GetHighestProgressSource, and
GetEarliestSource. Covers cases where each device wins, ties, and
missing/invalid data.
strings.Title has been deprecated since Go 1.18 because it does not
handle Unicode properly. Replace it with cases.Title from
golang.org/x/text which correctly handles language-specific title
casing. Applied to breadcrumb generation and document title formatting.
Simplify pointer creation in media scanner validation messages and worker
job timestamps by using inline new() instead of local variable + address-of.
In media_scanner.go this cleans up three validation error message returns
(manga/comics library format checks). In worker.go it simplifies StartedAt
and CompletedAt timestamp assignments.
Simplify pointer creation across kobo, koreader, and queue handlers by
replacing the two-step pattern (assign to local, then take address) with
inline new() calls. This reduces verbosity without changing behavior:
Before:
remaining := int(a - b)
pagesRemaining = &remaining
After:
pagesRemaining = new(int(a - b))
Covers page calculations, chapter/progress fields, UUID formatting,
and timestamp string conversions.
Replace the original 5-theme allowlist (light, sepia, dark, night,
high-contrast) with a richer 20-theme palette organized into tonal
families: neutrals (light, paper, slate, oled), warm tones (sepia,
parchment, warm, candlelight), cool tones (azure, sky, arctic, frost),
and evening tones (dusk, sunset, twilight, forest, moss, solarized).
The backend validator in UpdateSettings now accepts all 20 theme names,
and the frontend Tailwind build is updated to include the new theme CSS
variables and preflight reset.
Major rewrite of the web reader to properly interface with
@bookhoard/foliate-js, replacing the abandoned panel-detection
architecture with direct pan and zoom support built into the
foliate-js FixedLayout renderer.
Template (reader.templ):
- Fix critical bug: x-init config was using literal strings
'{ readerData.X }' inside a quoted attribute, which templ
treated as raw text and never interpolated. Values were never
actually passed to JavaScript. Now uses fmt.Sprintf() with
templ's expression attribute syntax ={ }.
- Pass fileUrl from server so foliate-js can open books directly.
- Redesign bottom bar with foliate-js parity: left/right navigation
buttons, progress slider with tick marks, and zoom controls
(zoom out, percentage display, zoom in, magnifier, pan/select
mode toggle for PDFs).
- Remove panel editor button and enablePanelDetection config.
- Add SVG icon styles for consistent reader controls.
Go types (templates/types.go):
- Expand ReaderMetadata with FormatGroup, MangaType,
ReadingDirection, FileURL, and LibraryID fields needed by
the reader frontend.
Router (internal/router/reader.go):
- Populate new ReaderMetadata fields from database values.
- Construct FileURL from library ID and file path for the
/uploads/library-{id}/* file serving route.
Reader JS (reader.ts):
- Full rewrite modeled on foliate-js Reader class, adapted for
Alpine.js. Opens books via view.open(fileUrl), accesses
view.renderer for zoom/pan/navigation, and wires up keyboard
shortcuts (+/-/0 for zoom, arrows for nav, Escape for magnifier).
- Uses view.isFixedLayout instead of importing FixedLayout class,
avoiding a TypeScript module resolution issue with the Vite alias.
Settings manager (settings-manager.ts):
- Remove dependency on deleted ReaderContext event bus.
- Export loadSettings/saveSettings/syncSettings directly as
standalone async functions.
Cleanup:
- Delete reader-context.ts and reader-events.ts (over-engineered
event system replaced by direct function calls).
- Remove panel_zoom_enabled from ReaderSettings type.
Apply Go 1.18+ language features and modern style:
internal/services/collection_service.go:
- Use map[string]any instead of map[string]interface{} (Go 1.18+)
- Use range clause with single variable for iteration-only loops
- Replace if-else chains with switch statements for better readability
- Remove explicit type initialization for zero values
internal/services/filters.go:
- Add Err prefix to custom error variable for error naming convention
internal/router/library.go:
- Use cfg.ProcessingIssuesHandler instead of local processingIssuesHandler variable
- Ensures proper dependency injection through router config
These changes follow current Go best practices and improve code readability.
Add frontend route /admin/libraries/:id/issues to display processing issues
management page for a specific library.
internal/router/frontend.go:
- Register GET /admin/libraries/:id/issues with admin middleware
- Fetch processing issue stats from database
- List processing issues for the library
- Convert database models to template types
- Render AdminProcessingIssues template with issues and stats
This provides the admin UI for viewing and managing processing errors that
occur during media scanning and import workflows.
Wire up the ProcessingIssuesHandler throughout the application:
cmd/server/main.go:
- Remove obsolete commented-out getTemplateUserWithTheme function
- Instantiate ProcessingIssuesHandler with database queries
- Add handler to router Config (with field alignment cleanup)
internal/router/router.go:
- Add ProcessingIssuesHandler field to router Config struct
- Reformat Config struct for better field alignment
This enables the processing issues API endpoints for listing and getting
statistics about issues within libraries, integrated with the admin UI.
Fix ResolveProcessingIssue and DeleteProcessingIssue methods to use proper
parameter structs instead of individual arguments.
Changes:
- ResolveProcessingIssue: Use database.ResolveProcessingIssueParams struct
with ID and MediaItemID fields instead of separate arguments
- DeleteProcessingIssue: Wrap issueID in pgtype.UUID struct
- Use map[string]any instead of map[string]interface{} for JSON responses
These changes align with the sqlc-generated database interface and ensure
type-safe parameter passing to the database layer.
Add Go package documentation comments to clarify the purpose and scope of:
- internal/handlers/: HTTP request/response handlers for authentication,
libraries, media items, reading, collections, dashboards, devices,
analytics, and system features
- internal/services/: Core business logic layer including media scanning,
library management, search, analytics, and conversion services
These doc comments improve code discoverability and help developers understand
the architectural separation between HTTP handling (handlers) and business
logic (services).
- Add ProcessingIssuesHandler with List and GetStats methods
- Add AdminProcessingIssues template for issues dashboard
- Display error/warning/info stats cards
- Sort issues by severity and creation date
- Add dismiss functionality for warnings and info items
- Add navigate to media item functionality
- Show issue type, description, and media details
- Add GET /admin/libraries/:id/issues/list for listing issues
- Add GET /admin/libraries/:id/issues/stats for issue statistics
- Integrate processing issues handler with library routes
- Add shouldEnablePanelDetection to determine when to enable panel detection
- Enable for manga/comics libraries with fixed_layout or comic_archive formats
- Fetch library type info using GetLibraryWithType query
- Pass panel detection config to reader initialization
- Include format_group, manga_type, and reading_direction in reader response
- Add ProcessingIssues model struct
- Update Querier interface with processing issues methods
- Add generated query implementations for CreateProcessingIssue, ListProcessingIssuesByLibrary, GetProcessingIssueStats, ResolveProcessingIssue, DeleteProcessingIssue
- Add GetLibraryWithType query for fetching library with type information
- Add CreateProcessingIssue with upsert for recording/renewing issues
- Add ListProcessingIssuesByLibrary with severity ordering and media item details
- Add GetProcessingIssueStats for error/warning/info counts
- Add ResolveProcessingIssue for marking issues as resolved
- Add DeleteProcessingIssue for removing resolved issues
- Add GetLibraryWithType for fetching library with type info for validation
Remove unused Epubcfi and Percentage fields from UpdateReadingProgressParams
struct to align with the new foliate-js based reader implementation.
The foliate-js library handles CFI tracking and percentage calculation
internally, so these parameters are no longer needed in the update API.
The reader now relies on foliate-js's built-in progress tracking mechanisms.
This change aligns the database layer with the foliate-js integration completed
in commit c7a9098 (feat: Replace foliate-js submodule with npm git dependency).
Changes:
- Remove Epubcfi field from UpdateReadingProgressParams struct
- Remove Percentage field from UpdateReadingProgressParams struct
- UpdateReadingProgress function now uses simplified parameter set
Refactor UpdateMediaReadingProgress handler to use UpdateUniversalProgress
database function, providing comprehensive progress tracking capabilities
including device sync information, viewport data, reading mode, and scroll
position for enhanced cross-platform reading synchronization.
Enhance reading progress tracking to support EPUB-specific location data:
- Add epubcfi field to store EPUB Canonical Fragment Identifier
- Add percentage field for normalized position across formats
- Update UpdateReadingProgress API handler to accept new fields
- Modify database queries to persist additional progress metadata
This enables precise position tracking in reflowable EPUB content
where page numbers are insufficient for accurate bookmarking.
- Add library_type_name to GetMediaItem handler response in media.go
- Remove empty LibraryName and LibraryTypeName fields from:
- ListMediaItemsRow in collections.go handler
- ListMediaItemsRow in dashboard_service.go
- These fields are now populated at the database level via trigger
The library_type_name is now automatically populated in the database
when a media item is created, so we remove the manual empty string
assignments and expose the actual value in the API response.
- Add library_type_name VARCHAR(50) column to media_items table in schema.sql
- Create PostgreSQL trigger 'set_library_type_name_on_insert' that automatically
populates library_type_name by joining libraries table with library_types on insert
- Add library_type_name parameter to CreateMediaItem SQL query
- Update Go models (models.go) to include LibraryTypeName field
- Add UpdateMediaItemChapterMetadata query method to querier.go
- Regenerate queries.sql.go with sqlc
This allows media items to store their library type (e.g., 'Books', 'Comics', 'Manga')
at the database level, enabling filtering and display without needing additional joins.
- templates/reader.templ: add script tag to load main.js
- Enables reader shell and Alpine components to initialize
- internal/handlers/reader.go: add GetMediaReadingProgress handler
- Backend now supports fetching reading progress via API
- GET /api/media-items/:id/progress returns current progress
- Fix function signatures in reader.go (c echo.Context -> c *echo.Context)
- Replace non-existent UserHasLibraryAccess with GetUserVisibleLibraries pattern
- Implement SSR reader route in router/reader.go with proper access control
- Add inline library access checking following existing codebase patterns
- Fix ReadingProgress struct to use LastReadAt instead of CreatedAt/UpdatedAt
- Ensure all reader endpoints use consistent library access validation
This provides the backend foundation for the reader feature with proper
access control and SSR rendering capabilities.
- Add panel_layout to getDefaultSettings() for dockable panels
- Remove template rendering from ShowReader (router handles SSR)
- Fix pgtype.Int4 marshaling to JSON (no explicit int conversion)
- Remove unused strings import from handlers
- Add modular dockable panel architecture with:
- Panel dock system (drag, lock, snap-back, window-shade)
- TOC, Settings, Navigator, Bookmarks as dockable components
- Lock toggle to prevent accidental moves
- Snap-back to last valid position if dropped in invalid area
- Per-user layout stored in reader_settings JSONB
- Update TypeScript types with PanelLayoutSettings and PanelState
- Add panel-dock-system.ts implementation to plan
- Add navigator-panel.ts for Affinity-style page navigation
- Update reader template with dockable panels and lock buttons
- Add Section 2.4 with database/Go implementation issues and fixes
- Document schema changes (DECIMAL -> REAL)
- Document all type fixes needed in reader.go
- Fix pgtype.UUID usage in test files by properly converting string UUIDs to pgtype.UUID
- Update numericToFloat to use pgtype.Float8 instead of pgtype.Numeric for DOUBLE PRECISION support
- Fix field name from WebURL to WebUrl to match current schema
These changes align with the recent community_rating type change to DOUBLE PRECISION
and ensure consistent type handling across the codebase.
Phase 6.1 implementation: Unit tests for metadata helper functions.
Test Coverage:
- TestNormalizeMangaType: Verify Manga field normalization to database enum values
(unknown, no, yes, yes_and_right_to_left)
- TestDetermineReadingDirection: Test reading direction computation heuristics
(explicit Manga field, Japanese language, webtoon/manhwa genre tags, Western default)
- TestNormalizeAgeRating: Verify age rating standardization
(Everyone, Teen, Mature, Adult with various input formats)
These tests ensure the helper functions correctly normalize ComicInfo.xml data
before storage in the database.
Relates to: Phase 6.1 unit testing
john-okeefe