Multi-file book ingest: a directory of ≥2 audio files (recursively —
disc-part folders like CD1/Disc 2 merge upward, siblings never merge)
ingests as ONE book whose file_path is the book-root directory; a zip
of audio tracks is its own book (file_path = the archive). Both write
the documented chapters abstraction {index,title,start,end} into
audio_metadata and add additive keys: layout ("folder"|"zip") and a
tracks[] manifest (per-track path, global-seconds start/end, duration,
size). Pre-grouping one-track strays under a book root are absorbed on
rescan (annotations cascade). Rescans take a stat-only fast path
(track count + total size + known durations unchanged → no re-probe —
a 1189-track book must not re-ffprobe every auto-scan); a zero-duration
book counts as changed so a ffprobe-equipped rescan fills durations in.
Folder-book roots count as present in the archive lifecycle pass, and
per-item rescan (RescanMediaItem) routes multi-file books through a
book-level refresh — the generic updater would read the root/zip as if
it were the book file and clobber format_group with "unknown".
Serving: GET /api/media-items/:id/chapters gains additive layout +
tracks keys (urls → the new endpoint). New GET
/api/media-items/:id/audio/:trackIndex streams one track — folder
tracks served from the real file (full Range via ServeContent), zip
tracks via a reopen-and-skip ReadSeeker over the entry (deflate has no
random access; acceptable at audio bitrates; stored entries stay
section readers). Both endpoints gate on library visibility (403 on
hidden libraries — family-lens rule). ServeFile and OPDS DownloadBook
refuse directory paths (folder books have no single-file download).
Web player (next commit) consumes the manifest; progress/bookmarks/
deep-links keep their global-seconds coordinates, so the server-side
progress model is untouched. sync learns FormatGroupAudiobook
(percentage passthrough in ConvertProgress); detectFormatType stops
reporting "unknown" for audio.
Metadata ingestion (Batch B) — read everything available, degrade
gracefully, documented precedence sidecars → folder structure →
embedded tags → filename: ffprobe tags are now a case-insensitive
lookup map (freeform keys pass through verbatim); composer fills
narrators (Audible's own convention, kept as ONE string — no reliable
multi-narrator delimiter); grouping + Audible freeform atoms
(SERIES/SERIES-PART/SUBTITLE/ASIN/ABRIDGED) fill series/index/
subtitle-join/ASIN/abridged (explicit sidecar abridged-false is
guarded); date/genre fall back to tags; album/album_artist/artist
fill title/author. Genre persists at create for all formats.
Embedded cover art (mp3 APIC, m4b covr, flac picture) extracts via one
optional ffmpeg call as a "<book>.cover.jpg" sibling — sidecar covers
win. CUE sheets (FILE-matched, INDEX 01 at 75fps) are the lowest-
priority chapter source under sidecars and embedded m4b chapters. A
lone m3u/m3u8 in a book root imposes playlist ordering + #EXTINF
names; (disc, track) tags reorder tracks when coverage is complete.
extractMetadata's default branch no longer pre-fills the filename as
title (tags must outrank it) — final filename fallbacks now exist on
BOTH the create and rescan paths (TestRescanKeepsFilenameTitle pins
that rescans cannot blank a title); tagless zip books take the zip's
own name, not their parent folder's.
Accepted containers widen to .m4b .m4a .mp3 .flac .zip .opus .ogg .oga
.aac .wav with real media types registered (audio/ogg per RFC 7845 for
Opus-in-Ogg) — all browser-playable; .wma deliberately excluded.
Tests: unit tests for CUE parsing, playlist hints, tag fallbacks
(synthetic ffprobe reports — ffmpeg cannot write Audible freeform
atoms), ordinal parsing, tag ordering, natural sort, grouping rules;
integration suite (11 tests) over the live dev DB: folder/zip ingest +
idempotency, stray absorption, chapters manifest, per-track streaming
with Range, hidden-library 403s, embedded covers + sidecar-wins, CUE
chapters, embedded m4b chapters, playlist ordering, tag fallbacks, all
five new containers with content-type assertions, rescan title pin.
Fixture media is real committed-format binary content living untracked
under internal/services/testdata/ (gitignored by policy; recipes in
docs/contributing/development.md → Media fixtures). Test image now
ships ffmpeg to match the runtime's audio toolchain.
Phase 1 (foundation): 'audiobooks' library type seeded (schema + DB);
audio_metadata JSONB on media_items (the kobo_metadata precedent —
only audio paths read it); reading_progress.audio_offset_seconds for
the honest {chapter, offset} locator; extensions/mimes registered for
ALL breadth formats (m4b/m4a/mp3/flac — deliberate per the DOC/LIT
doctrine, m4b scans richly first); classifyFormatGroup 'audiobook'
case (gotcha 39 recomputes on rescan); Go mime table extended at boot
so media elements accept the streams; ffmpeg/ffprobe in the image
(8.1.2 — also the xHE-AAC identification tripwire).
Phase 2 (ingest): sidecar-first (Libation/ABS metadata.json —
narrators/ASIN/curated chapters), ffprobe fallback/filler (duration,
bitrate, codec/profile, embedded chapters). Fixed three real
parser brittlenesses the Libation sidecar exposed: series accepts
string OR array-of-objects, publishedYear accepts number OR string,
ffprobe chapter timings accept number OR string (flexFloat). Folder-
structure fallback now runs on the UPDATE path too (rescans were
clearing sidecar-absent series/author); audio facts re-derive on
every rescan (gotcha 39).
Phase 3 (playback + sync): GET /api/media-items/:id/chapters (the
format-abstraction list); streaming rides the existing cookie-authed
Range /uploads path; PUT progress carries audio_offset_seconds +
elapsed_seconds (honest wall-clock playing time) — the reading-
history capture prefers player-reported elapsed over the inter-push
delta, same 2h ceiling; reading_mode='audio' marks the row.
Phase 4 (web player): dedicated audio reader page (foliate not
involved) — cover, chapter title, whole-book scrubber, skip ± (chips
1/5/10/15/30s, default 10s), speed (0.75×–2×), chapter list with
jump-to, Media Session lockscreen/notification transport, periodic
playing-only progress pushes, position restore on open.
Verified live: ingest of the Libation sample (Gardens of the Moon —
title/author/series/cover/date from sidecar+folders, 34 chapters,
26h duration, aac/LC); chapters endpoint; the reader route serving
the player; playback with position restore (0:57 round-trip);
progress rows in audio mode; wall-clock session rows in
reading_history.
The version tag previously existed only in git and the Docker image tag —
the running app had no way to report what build it was. Version is now
injected at build time via -ldflags into internal/version.Version (defaults
to "dev" for local builds), passed by the release workflow as the
APP_VERSION Docker build arg from the pushed tag.
Surfaced in Admin → Settings → About (extensible card for future rows like
disk space) and in the /health JSON response, so deployments can be
verified with curl alone.
The test-runner stage copied only /app from the builder, but the Go module
cache lives in /go/pkg/mod (GOMODCACHE) — the cache mounts used by the
builder target /root/go/pkg/mod, which the go tool ignores, so those mounts
never held anything. Every `compose run tests` / make test-integration
invocation therefore re-downloaded all dependencies from the network,
making integration runs slow and timeout-prone.
The builder now materializes /go/pkg/mod into an image layer
(go-module-cache) after the binary build, and the test-runner stage
restores it to /go/pkg/mod before running tests. Test-only stage; the
production final stage is unaffected.
extractPDFCover previously only saved embedded raster images from page 1,
so vector/text-first-page PDFs (e.g. InDesign exports like Data Structures
the Fun Way) ended up with no cover and a dashboard placeholder. When no
embedded image is found it now falls back to rendering page 1 with
pdftoppm (poppler-utils), saving the same {pdf}.cover.jpg sidecar.
Also adds MediaScanner.RescanMediaItem, which re-extracts metadata for a
single media item (resolving its on-disk path from library folders) so
previously imported books can backfill covers without a full force rescan.
Dockerfile installs poppler-utils in the final and test-runner stages.
Add --mount=type=cache for Go build cache to speed up repeated builds.
Remove the -a flag which forces full rebuilding of all packages and is
unnecessary when using cache mounts.
npm ci requires a package-lock.json and installs exactly what it specifies.
npm install resolves from package.json directly, making the Docker build
fully self-contained — you can clone the repo and run docker build with no
local Node tooling or pre-existing lockfile.
Newer templ generates ResolveAttributeValue calls that don't exist in
v0.3.1001 runtime, causing Docker build failures ("undefined:
templ.ResolveAttributeValue"). Pin templ CLI version in Dockerfile to
match go.mod instead of using @latest.
Also updated local templ CLI to v0.3.1020 to match.
The builder stage was previously bumped to Go 1.26 but the test-runner stage
remained on Go 1.25, causing 'go.mod requires go >= 1.26.0' errors during
test execution.
Go 1.26 introduced stricter validation of replace directives in dependency
go.mod files, which broke 'go install sqlc@latest' (sqlc v1.31.0 has replace
directives). Replace go install with a direct binary download from GitHub
releases, matching the existing kepubify download pattern.
Bump test-runner from golang:1.25-alpine to golang:1.26-alpine to match the
go.mod requirement.
- Modified package.json build:ts to copy htmx.min.js from node_modules to web/static/
- This fixes the 404 error for /static/htmx.min.js that occurred after ESBuild migration
- Added htmx.min.js to static files
See ESBUILD_MIGRATION_PLAN.md for migration context.
Remove minified JavaScript libraries that were previously downloaded during
postinstall (htmx.min.js, highlight.min.js, lunr.min.js, lunr-flex.min.js).
These are now bundled via esbuild from npm packages.
Update Dockerfile to remove the now-unnecessary postinstall npm script execution,
streamlining the container build process.
- Optimize Dockerfile layer caching with --mount=type=cache for Go modules and npm
- Reorganize Dockerfile layers for better cache hit rates
- Improve .dockerignore organization with categorized comments
- Add Makefile targets: up, down, rebuild-app, restart, ps
- Enhance Makefile help output with categorized sections
- TestMediaItemISBNNormalization was missing library folder creation
- Caused HTTP 400 errors when creating media items
- addFolderToLibrary call was accidentally removed from line 73
- Tests now properly create library with folder before adding media items
This fixes the root cause of ISBN normalization test failures where
media-item creation failed due to missing library folder requirement.
- Add test-runner stage to Dockerfile for isolated test execution
- Refactor Makefile test targets: separate unit and integration tests
- Unit tests now run on host (fast, no containers required)
- Integration tests run in containers matching production environment
- Add dedicated 'tests' service to docker-compose.yml
- Update test-integration target to use containerized test runner
- Improve service health checks and wait conditions
- Add test environment variables for consistent testing
This change separates unit tests (fast, local) from integration tests
(full environment, containerized) for better developer experience
and more reliable CI/CD pipelines.
- Renamed subdirectory INDEX.md files to section-specific names:
- user/INDEX.md → user/user-guide.md
- developer/INDEX.md → developer/development.md
- operations/INDEX.md → operations/operations.md
- contributing/INDEX.md → contributing/contributing.md
- developer/api/INDEX.md → developer/api/api-reference.md
- developer/api/collections/INDEX.md → developer/api/collections/collections-api.md
- Updated all internal links to use new filenames
- Updated navigation.go to skip subdirectory INDEX files from sidebar
- Added Dockerfile to include docs directory in container build
This fixes the issue where multiple 'INDEX' links appeared in the sidebar,
making navigation confusing. Now each section has a descriptive name.
- Install kepubify binary in Dockerfile for on-the-fly conversion
- Add conversion service with caching layer (24hr TTL)
- Support KEPUB downloads through OPDS endpoint
- Cache converted files to reduce processing overhead
- Add environment configuration for cache directory and tool path
- Reorder package installation to improve layer caching
- Add curl to runtime dependencies
- Use relative paths for sqlc and templ generation
- Copy static assets to final stage
- Improve build step organization
- Move migrations/ to database/schema/ for clarity on database schema definitions
- Move sqlc.yaml to internal/database/ to group with database code
- Move static/ to cmd/server/static/ to co-locate with server
- Update all configuration files and documentation
- Follow Go project conventions for better organization
- Rename project from 'bookmann' to 'shelf'
- Move all backend/ contents to root level (flatten structure)
- Update Go module name from 'bookmann' to 'shelf'
- Update all import paths to use new 'shelf' module
- Update Dockerfile to work without backend/ subdirectory
- Update docker-compose.yml to use new structure and rename containers
- Update .gitignore for new file paths
- Update README.md with new project name and structure
- Regenerate database code with new module imports
John O'Keefe
john-okeefe