Multi-file book ingest: a directory of ≥2 audio files (recursively —
disc-part folders like CD1/Disc 2 merge upward, siblings never merge)
ingests as ONE book whose file_path is the book-root directory; a zip
of audio tracks is its own book (file_path = the archive). Both write
the documented chapters abstraction {index,title,start,end} into
audio_metadata and add additive keys: layout ("folder"|"zip") and a
tracks[] manifest (per-track path, global-seconds start/end, duration,
size). Pre-grouping one-track strays under a book root are absorbed on
rescan (annotations cascade). Rescans take a stat-only fast path
(track count + total size + known durations unchanged → no re-probe —
a 1189-track book must not re-ffprobe every auto-scan); a zero-duration
book counts as changed so a ffprobe-equipped rescan fills durations in.
Folder-book roots count as present in the archive lifecycle pass, and
per-item rescan (RescanMediaItem) routes multi-file books through a
book-level refresh — the generic updater would read the root/zip as if
it were the book file and clobber format_group with "unknown".
Serving: GET /api/media-items/:id/chapters gains additive layout +
tracks keys (urls → the new endpoint). New GET
/api/media-items/:id/audio/:trackIndex streams one track — folder
tracks served from the real file (full Range via ServeContent), zip
tracks via a reopen-and-skip ReadSeeker over the entry (deflate has no
random access; acceptable at audio bitrates; stored entries stay
section readers). Both endpoints gate on library visibility (403 on
hidden libraries — family-lens rule). ServeFile and OPDS DownloadBook
refuse directory paths (folder books have no single-file download).
Web player (next commit) consumes the manifest; progress/bookmarks/
deep-links keep their global-seconds coordinates, so the server-side
progress model is untouched. sync learns FormatGroupAudiobook
(percentage passthrough in ConvertProgress); detectFormatType stops
reporting "unknown" for audio.
Metadata ingestion (Batch B) — read everything available, degrade
gracefully, documented precedence sidecars → folder structure →
embedded tags → filename: ffprobe tags are now a case-insensitive
lookup map (freeform keys pass through verbatim); composer fills
narrators (Audible's own convention, kept as ONE string — no reliable
multi-narrator delimiter); grouping + Audible freeform atoms
(SERIES/SERIES-PART/SUBTITLE/ASIN/ABRIDGED) fill series/index/
subtitle-join/ASIN/abridged (explicit sidecar abridged-false is
guarded); date/genre fall back to tags; album/album_artist/artist
fill title/author. Genre persists at create for all formats.
Embedded cover art (mp3 APIC, m4b covr, flac picture) extracts via one
optional ffmpeg call as a "<book>.cover.jpg" sibling — sidecar covers
win. CUE sheets (FILE-matched, INDEX 01 at 75fps) are the lowest-
priority chapter source under sidecars and embedded m4b chapters. A
lone m3u/m3u8 in a book root imposes playlist ordering + #EXTINF
names; (disc, track) tags reorder tracks when coverage is complete.
extractMetadata's default branch no longer pre-fills the filename as
title (tags must outrank it) — final filename fallbacks now exist on
BOTH the create and rescan paths (TestRescanKeepsFilenameTitle pins
that rescans cannot blank a title); tagless zip books take the zip's
own name, not their parent folder's.
Accepted containers widen to .m4b .m4a .mp3 .flac .zip .opus .ogg .oga
.aac .wav with real media types registered (audio/ogg per RFC 7845 for
Opus-in-Ogg) — all browser-playable; .wma deliberately excluded.
Tests: unit tests for CUE parsing, playlist hints, tag fallbacks
(synthetic ffprobe reports — ffmpeg cannot write Audible freeform
atoms), ordinal parsing, tag ordering, natural sort, grouping rules;
integration suite (11 tests) over the live dev DB: folder/zip ingest +
idempotency, stray absorption, chapters manifest, per-track streaming
with Range, hidden-library 403s, embedded covers + sidecar-wins, CUE
chapters, embedded m4b chapters, playlist ordering, tag fallbacks, all
five new containers with content-type assertions, rescan title pin.
Fixture media is real committed-format binary content living untracked
under internal/services/testdata/ (gitignored by policy; recipes in
docs/contributing/development.md → Media fixtures). Test image now
ships ffmpeg to match the runtime's audio toolchain.
The search feed carried a ~55-line inline copy of the acquisition-entry
skeleton, over a DIFFERENT sqlc row type (SearchMediaItemsRow vs
ListMediaItemsByLibraryRow) — the row-type split is why it stayed a
twin. The shared builder now takes opdsEntryInput, the narrow
ten-field subset the feed actually renders (deliberately not a row
converter), and acquisitionEntryOpts makes the two legitimate
differences explicit instead of incidental:
- labelLibraries: unscoped search labels entries with their library.
- includePdfAlternate: catalogs offer the PDF alternate for reflowable
books; search historically does not. Kept as-is — making search match
catalogs would be a behavior change, flagged in the opts doc.
The four OPDS behaviors tests are green in the test-runner container.
Flagged, not from this change: TestOPDSLibraryFolders fails identically
with the change stashed — pre-existing rot worth its own look (it never
ran in the hardening batteries because the -run patterns didn't match
it).
The audit showed only 5 of ~11 time-display surfaces converted to the
user's timezone; everything else rendered pgx-decoded UTC. The server
owns conversion — clients (web/app/KOReader) receive display-ready
strings or machine-readable instants and never convert.
- Progress page: LastUpdated formatted in the requesting user's tz
(was raw UTC).
- Analytics: date-range defaults, day buckets, most-active-day, device
last-sync, and popular-book last-read all computed in the user's tz —
evening sessions no longer land on the next UTC day. Also fixed the
end-date boundary: the query bounds with <= and the end bound was
midnight, so the end date's own sessions (including 'today') never
appeared.
- Queue page: new created_at_display field (admin tz) rendered by the
template; created_at stays RFC3339 for API consumers.
- Admin hash-conflicts SSR bridge: detected date rendered in the
admin's tz.
- OPDS: .UTC() before the Z-suffixed feed timestamps (honesty fix —
correct today only because the session zone happens to be UTC).
- KOReader annotation datetimes deliberately untouched (device owns its
display).
The device catalog flattened every visible library into one list, so
duplicate copies of the same book each got an entry and search had no
library context. The root feed is now a standard OPDS 1.1 navigation feed
that mirrors the web UI's library model:
- Root /catalog: an "All Books" entry first (the previous flat cross-
library behavior, also still served at ?all=1 for clients that want
the single flat list), followed by one folder entry per visible
library with live book counts from GetVisibleLibraryMediaCounts.
- New GET /library/:libraryId/catalog: acquisition feed scoped to one
visible library (403 when the device owner cannot see it), paginated,
with an up-link back to the root.
- Scoped search: each library feed's rel="search" OpenSearch template
pins &library_id=<id>, so opening search from inside a library folder
searches only that library — clients substitute only {searchTerms},
so no client-side changes are required. Root search stays global.
- Global search entries now carry the owning library as a category (and
as a fallback summary when the book has no description), so duplicate
copies are distinguishable in unscoped result lists.
The acquisition entry builder is extracted into addAcquisitionEntries and
shared by the all-books and per-library feeds. bookhoard.koplugin needs
no changes: it only registers the root URL, and KOReader's stock OPDS
client renders navigation feeds natively.
Tests: TestOPDSLibraryFolders covers the nav-feed shape, flat ?all=1
mode, scoped catalog isolation, scoped/global search behavior, and 403s
for libraries outside the device's visibility. Library names avoid the
word "test" on purpose — setupDeviceTest re-runs setupTestServer's
setup-time cleanup, which deletes every library whose name contains it.
DownloadBook populated fileSha256 only for the kepub and pdf format
branches, so the default EPUB download never emitted the
X-Bookhoard-SHA256 response header - the hash was only available in
the feed metadata, not on the download response itself.
Populate it from mediaItem.FileSha256 in the default branch so every
download response carries the canonical primary-format hash. Clients
that capture response headers at download time now learn the hash
regardless of which format they requested.
Move three more hardcoded values behind the settings registry. All
apply immediately on the next request (no restart needed).
device_auth.go:
- DeviceAuthMiddleware reads per-route device rate limits (sync /
progress / metadata per minute) from the registry on each
authenticated request via a rateLimitConfig() helper, falling back to
the Default* constants when no registry is wired.
- The X-RateLimit-Limit response header previously hardcoded "60" for
every request type; it now reflects the actual configured limit for
the request type via rateLimitForRequestType().
opds.go:
- Default (50) and maximum (200) OPDS page sizes come from the
registry's OpdsDefaultPageSize()/OpdsMaxPageSize() instead of inline
literals, so catalog pagination can be tuned without a redeploy.
conversion_service.go:
- The 24h kepub cache lifetime is read from the registry via a
cacheTTL() helper (was a bare 24 * time.Hour literal in the
constructor). The field default is retained for tests that construct
the service directly.
- conversion_service_test.go updated to assert both the field default
and the cacheTTL() accessor return 24h.
Three bugs fixed:
1. Schema seeded base_url with fake placeholder 'bookhoard.example.com'.
Removed seed; startup now seeds from BASE_URL env var only if DB row
is empty (admin changes persist across restarts). One-time UPDATE
clears the placeholder in existing installs.
2. config.GetBaseURL() had a broken type assertion (local SystemConfigRow
vs database.SystemConfig) that always failed, returning . Admin panel
showed env var fallback instead of actual DB value. Fixed with a
function-type getter that properly wraps the DB query.
3. OPDS handler read base_url only from DB with no fallback. When DB had
the placeholder, all feed links pointed to an unreachable domain,
breaking KOReader search/download. Added deriveBaseURL() helper that
falls back to the request Host/scheme when DB value is empty.
Setup gate improvements:
- isSetupComplete now requires both admin user AND non-empty base_url
- Setup middleware no longer exempts all /api/ routes; only allows
/api/auth/register, /api/auth/login, /api/system/config before setup
is complete. All other API routes get 503.
- Cache invalidated when base_url is saved via admin settings
Dev workflow:
- New bruno/NewDevDBSetup/SetBaseUrl.yml for dev DB setup
- NewDB.sh runs SetBaseUrl between RegisterUser and CreateEbookLibrary
The device catalog feed was unusable on paged OPDS clients such as
KOReader: it sliced results into pages but never advertised how to reach
the next page, so clients could only ever fetch the first page (~50 books)
and could not search the catalog.
GetDeviceCatalog:
- Emit the full set of OPDS pagination link relations (self, start, first,
previous, next, last) pointing at catalog?page=N&per_page=M, with the
device auth token appended for path-based auth.
- Emit OpenSearch totalResults/itemsPerPage/startIndex metadata.
- Point rel=search at the OpenSearch description (correct MIME type).
SearchDeviceCatalog now branches on the q parameter:
- No q: return an OpenSearch description document whose Url template
contains the {searchTerms} placeholder, so clients can formulate a query.
- With q: return the existing acquisition results feed, now including
totalResults.
A pure addCatalogPaginationLinks helper holds the page/URL logic so it can
be unit tested without a database. New handler tests cover middle/first/
last/single/empty pages (correct presence of next/previous) and token
appending.
Ordering is intentionally left unchanged (created_at DESC, grouped by
library).
The OPDS device catalog previously hard-coded every acquisition link as
application/epub+zip and always offered kepub/pdf alternate links, which
is wrong for comic archives (cbz/cbr/cb7/cbt) and other non-epub media.
- Resolve the acquisition mime type from the media item's stored
mime_type (falling back to format_mimetype, then epub) instead of
assuming epub
- Only offer reflowable conversions (kepub for kobo, pdf) for ebooks;
comic archives are served as-is in their native format
- Derive the native format label (epub/pdf/cbz/cbr/...) from the file
path in ListFormats rather than always reporting epub
- Add resolveMimeType, isComicArchive, and formatLabelFromPath helpers
- Add <title> and <author><name> elements to Atom feed for compatibility
- Remove doubled /opds/opds/ path prefix in feed URLs
- Include ?token= auth param on all OPDS URLs
- Serve kepub links only for Kobo devices
- Fix URL construction for entries and acquisitions
Remove redundant type conversions that Go 1.26 makes unnecessary or that
were already no-ops:
- uuid.UUID(x.Bytes) → x.Bytes (uuid.UUID is [16]byte, same as pgtype UUID Bytes)
- pgtype.UUID{Bytes: [16]byte(u), Valid: true} → pgtype.UUID{Bytes: u, Valid: true}
- (*time.Time)(&x.Time) → &x.Time
- json.RawMessage(x) → x where x is already []byte
- []byte(stringVal) → stringVal where []byte is expected
- int()/int64()/byte() casts on values already of the target type
- Decompressor(fn) → fn (type is identical)
Handle previously ignored error returns:
- collections.go: check json.Unmarshal error in GetCollection
- conversion_service.go: check fileSize.Scan() error
- app_test.go: check app.Shutdown() error in benchmark
Refactor configuration to use one source of truth for base URL
- Add GetBaseURL() to config package: queries system_config table first,
falls back to BASE_URL env var
- Update SidecarHandler to accept config and use single base_url
- Compute opds/api paths from base_url instead of storing separately:
- OPDS: base_url + /opds
- API: base_url + /api
- Device Sync: base_url + /api/sync
- Simplify OPDSHandler.getBaseURLs() to compute opds path
- Remove need for separate opds_base_url and api_base_url columns
Previously the system stored three separate URL config values that were
usually the same domain with different paths. Now store only base_url
and compute the paths, eliminating configuration redundancy.
Update the DownloadBook function to properly resolve media file paths using
the LibraryService.ResolveMediaPath method instead of directly accessing the
FilePath field. This ensures correct file resolution after the migration to
relative path storage.
The change affects three code paths in the download handler:
- KEPUB conversion path
- Direct file serve path (non-EPUB with conversion service)
- Default EPUB path
Error handling added to return 404 when path resolution fails, preventing
potential errors when accessing non-existent files.
This fixes potential file access issues after the relative path storage
implementation.
- Add libraryService dependency to CollectionHandler and OPDSHandler for centralized path resolution
- Create internal/utils/mediaurl.go with ResolveMediaURL() function as single source of truth
- Update GetMediaItem and ListMediaItems handlers to return resolved URLs in API responses
- Update collection handlers (GetCollection, TestRules, PreviewCollection) to use resolved cover URLs
- Update progress handler (GetAllProgress) to use resolved cover URLs
- Add library_id to GetCollectionItems SQL query to enable URL resolution
- Refactor media scanner to store relative paths instead of absolute filesystem paths
- Add ResolveMediaPath() to LibraryService for resolving relative paths to absolute paths
- Add ServeFile endpoint at /uploads/library-:id/* for authenticated file serving
- Add MimeTypes map to library_service.go for consistent MIME type handling
- Update DownloadBook handler to use resolved filesystem paths
- Add getRelativePath() helper to MediaScanner for converting absolute to relative paths
- Use strings.EqualFold for case-insensitive path comparisons in zip extraction
This change enables the application to work with relative paths stored in the
database, making it portable across different server environments while
maintaining backward compatibility with existing absolute paths.
- Integrate conversion service with OPDS handler
- Convert EPUB to KEPUB format on download request
- Cache converted files to reduce processing time
- Support per-device catalog with format availability
- Maintain backward compatibility with existing downloads
- Add OPDS handler for device catalog and book downloads
- Add collections handler for collection CRUD
- Add book matching service for cross-device book linking
- Add sidecar handler for Kobo metadata sync
- Add sync handler for device synchronization
John O'Keefe
john-okeefe