Compare commits

..
8 Commits
Author SHA1 Message Date
John O'Keefe ee98c0fb3b docs(dev): test-running policy + media-fixture rules; ignore testdata
Release / build-and-push (push) Successful in 3m29s
Integration tests require the containerized dev DB and bind-mounted
paths — a bare host `go test ./cmd/server/tests` fails on database
auth on every machine not running the compose stack (the old doc
prescribed exactly that flow). development.md now documents the
container runner and the iteration rule: run ONLY your own tests via
a filtered container invocation while working, and reserve full-suite
runs for pre-commit validation — several tests read live shared state
(system_settings, real library folders, the periodic backup scan's
websocket broadcasts) and flap on data or timing a change never
touched.

Binary media fixtures are never committed: internal/services/
testdata/ is gitignored by policy (the audio metadata tests read
real-format binaries that ffmpeg cannot synthesize — Audible freeform
atoms — so they live untracked on the machines that run the suite).
The new "Media fixtures" section carries the ffmpeg recipes for every
supported container, tag writing, embedded covers and ffmetadata
chapters, plus the manual end-to-end recipe for the live system
(one book per subfolder — a directory with ≥2 audio files ingests as
one folder book).
2026-10-19 07:54:55 -04:00
John O'Keefe 79ce5ab89d feat(web): multi-track audiobook player — per-track src switching, global-seconds position
The audio reader pointed <audio src> at ONE file — correct for m4b but
impossible for folder-of-tracks and zip books (the element holds a
single source). The player now generalizes to a track list built from
the /chapters manifest: single-file books are exactly one track whose
url is the whole file (zero behavior change — same element, same URL
flow), multi-file books swap src at track boundaries.

Mapping helpers (exported, unit-testable): trackIndexFor resolves a
global position to the owning track, localOffsetFor/globalSecondsFor
convert between global seconds — the coordinate system of chapters,
bookmarks and audio_offset_seconds — and (track, local offset). Every
consumer keeps global semantics: scrubber, skip ±N, bookmark saves,
?loc=audio: deep links and progress pushes are byte-identical payloads
server-side; the server contract does not move.

Mechanics: setActiveTrack swaps src and queues the position for
loadedmetadata (setting currentTime before the new metadata loads
resets it); ended advances to the next track and keeps playing;
timeupdate reports globalPosition(); playbackRate/volume re-apply
after swaps; src is deferred until the manifest loads (data-file-url
is a directory/zip root for multi-file books, not playable) with the
catch branch falling back to it for offline single-file starts.
2026-10-19 07:54:45 -04:00
John O'Keefe 6216bd9bcf feat(audiobooks): multi-file books (folder-of-tracks + zip) + exhaustive metadata ingestion
Multi-file book ingest: a directory of ≥2 audio files (recursively —
disc-part folders like CD1/Disc 2 merge upward, siblings never merge)
ingests as ONE book whose file_path is the book-root directory; a zip
of audio tracks is its own book (file_path = the archive). Both write
the documented chapters abstraction {index,title,start,end} into
audio_metadata and add additive keys: layout ("folder"|"zip") and a
tracks[] manifest (per-track path, global-seconds start/end, duration,
size). Pre-grouping one-track strays under a book root are absorbed on
rescan (annotations cascade). Rescans take a stat-only fast path
(track count + total size + known durations unchanged → no re-probe —
a 1189-track book must not re-ffprobe every auto-scan); a zero-duration
book counts as changed so a ffprobe-equipped rescan fills durations in.
Folder-book roots count as present in the archive lifecycle pass, and
per-item rescan (RescanMediaItem) routes multi-file books through a
book-level refresh — the generic updater would read the root/zip as if
it were the book file and clobber format_group with "unknown".

Serving: GET /api/media-items/:id/chapters gains additive layout +
tracks keys (urls → the new endpoint). New GET
/api/media-items/:id/audio/:trackIndex streams one track — folder
tracks served from the real file (full Range via ServeContent), zip
tracks via a reopen-and-skip ReadSeeker over the entry (deflate has no
random access; acceptable at audio bitrates; stored entries stay
section readers). Both endpoints gate on library visibility (403 on
hidden libraries — family-lens rule). ServeFile and OPDS DownloadBook
refuse directory paths (folder books have no single-file download).

Web player (next commit) consumes the manifest; progress/bookmarks/
deep-links keep their global-seconds coordinates, so the server-side
progress model is untouched. sync learns FormatGroupAudiobook
(percentage passthrough in ConvertProgress); detectFormatType stops
reporting "unknown" for audio.

Metadata ingestion (Batch B) — read everything available, degrade
gracefully, documented precedence sidecars → folder structure →
embedded tags → filename: ffprobe tags are now a case-insensitive
lookup map (freeform keys pass through verbatim); composer fills
narrators (Audible's own convention, kept as ONE string — no reliable
multi-narrator delimiter); grouping + Audible freeform atoms
(SERIES/SERIES-PART/SUBTITLE/ASIN/ABRIDGED) fill series/index/
subtitle-join/ASIN/abridged (explicit sidecar abridged-false is
guarded); date/genre fall back to tags; album/album_artist/artist
fill title/author. Genre persists at create for all formats.
Embedded cover art (mp3 APIC, m4b covr, flac picture) extracts via one
optional ffmpeg call as a "<book>.cover.jpg" sibling — sidecar covers
win. CUE sheets (FILE-matched, INDEX 01 at 75fps) are the lowest-
priority chapter source under sidecars and embedded m4b chapters. A
lone m3u/m3u8 in a book root imposes playlist ordering + #EXTINF
names; (disc, track) tags reorder tracks when coverage is complete.
extractMetadata's default branch no longer pre-fills the filename as
title (tags must outrank it) — final filename fallbacks now exist on
BOTH the create and rescan paths (TestRescanKeepsFilenameTitle pins
that rescans cannot blank a title); tagless zip books take the zip's
own name, not their parent folder's.

Accepted containers widen to .m4b .m4a .mp3 .flac .zip .opus .ogg .oga
.aac .wav with real media types registered (audio/ogg per RFC 7845 for
Opus-in-Ogg) — all browser-playable; .wma deliberately excluded.

Tests: unit tests for CUE parsing, playlist hints, tag fallbacks
(synthetic ffprobe reports — ffmpeg cannot write Audible freeform
atoms), ordinal parsing, tag ordering, natural sort, grouping rules;
integration suite (11 tests) over the live dev DB: folder/zip ingest +
idempotency, stray absorption, chapters manifest, per-track streaming
with Range, hidden-library 403s, embedded covers + sidecar-wins, CUE
chapters, embedded m4b chapters, playlist ordering, tag fallbacks, all
five new containers with content-type assertions, rescan title pin.
Fixture media is real committed-format binary content living untracked
under internal/services/testdata/ (gitignored by policy; recipes in
docs/contributing/development.md → Media fixtures). Test image now
ships ffmpeg to match the runtime's audio toolchain.
2026-10-19 07:54:28 -04:00
John O'Keefe 24228e7147 test(scan): cancellation endpoint semantics and mid-run safety
- endpoint semantics: non-admin cancel is 403, unknown job is 404
- mid-scan safety: start a scan of 1200 unparseable .epub files and
  cancel it, then pin the invariant that no media item is marked
  missing or archived. The canary item's file is lexically LAST in
  the folder, so an abort that (incorrectly) ran the archive pass
  would treat it as unvisited and trip the assertion. Assertions
  hold under every interleaving - whether the cancel lands mid-walk
  or the scan already finished - so the test cannot flap against
  timing.
- re-cancelling a terminal job is a 409

The test library is created through the shared
createTestLibraryWithFolder helper, which registers API-delete
cleanup so runs leave no residue in the dev database.

Worker-level unit tests live in internal/services/worker_cancel_test.go
(previous commit).
2026-10-18 18:02:45 -04:00
John O'Keefe e1153b2c4b feat(admin): cancel buttons for in-flight scans + cancelled state UI
- per-row Cancel button in the scan-progress card; listeners are bound
  after the TS island renders its rows (module scope has no inline
  handlers), and the button is removed once its job turns terminal
- cancel-all icon button in the card header: resolves the server's
  active job list first, so it also covers scans this page never
  started (e.g. started before a refresh), then cancels each one
- polling treats cancelled as terminal, averages overall progress
  only over jobs that can still advance (so the bar can reach 100%),
  and the results panel reports cancelled scans instead of claiming
  "Scan Complete!"
- sidebar spinner resets on the scan_cancelled websocket event (same
  quiet handling as scan_error)
- setup wizard shows "Cancelled" as a terminal status in its poller

No new innerHTML assignment sites - rendering-ownership baseline
unchanged.
2026-10-18 18:02:34 -04:00
John O'Keefe 5b81e19f2f feat(scanner): POST /api/scanner/cancel/:jobId (admin)
Cancels a pending or running scan job. Responses: 200 with the job id
and status on acceptance, 404 for unknown jobs, 409 for jobs already
in a terminal state (completed/failed/cancelled).

Additive route in the existing admin scanner group; no contract pins
affected.
2026-10-18 18:02:25 -04:00
John O'Keefe 30ecdd4bf3 feat(scanner): ScanFolders honors its context; cancelling never archives
ScanFolders never checked its context, so a cancelled job's walk ran
to completion no matter what the worker's cancel did.

- ctx checks at every stage boundary (file-counting pass, per folder,
  before and during the archive pass) and inside the walk callback
  (fs.SkipAll): the file currently being processed finishes, then the
  scan aborts with context.Canceled, which processJob classifies as a
  cancelled job rather than a failed one
- the archive/purge pass is skipped entirely when the scan was
  cancelled. It only visits what the walk visited, so after a partial
  walk the unvisited (but on-disk) files would be treated as missing,
  archived, and eventually purged. Cancelling is a UI convenience and
  must never destroy library data.
2026-10-18 18:02:19 -04:00
John O'Keefe b2220756e2 feat(worker): real per-job cancellation for the background job pool
CancelJob existed but was dead code: it only flipped a status in the
in-memory results map - it never cancelled anything, and processJob's
unconditional completion write overwrote the marker anyway.

- derive a per-job context in processJob (parent is the handler ctx
  for manual scans, Background for auto-scans) and register its
  cancel func under the job id; previously every manual scan shared
  the handler's process-lifetime context, so no targeted cancel was
  possible without killing everything
- skip execution at dequeue for jobs cancelled while still queued
- record a pending JobResult on BOTH enqueue paths (Enqueue used by
  auto-scans, EnqueueJob used by the API): queued jobs were invisible
  to status polling (404 until a worker picked them up) and could not
  be cancelled before execution started
- cancelled jobs keep their last real progress instead of a fake 100%
  and drop the meaningless "context canceled" error text
- broadcast the new scan_cancelled websocket event for cancelled scan
  jobs instead of scan_complete, so clients reset their scanning
  indicators rather than reading the abort as a finished scan
- sentinel errors (ErrJobNotFound, ErrJobNotCancellable) so HTTP
  handlers map outcomes to 404/409 without string matching

Unit tests (worker_cancel_test.go) pin: queued-cancel never executes,
running-cancel invokes the context, error mapping, pending recording
on both enqueue paths.
2026-10-18 18:02:11 -04:00
41 changed files with 4303 additions and 116 deletions

No files matched your search

+4
View File
@@ -82,3 +82,7 @@ session-*.md
# Local prod-export scratch: personal annotation/reading data, never commit
/export/
# Binary media fixtures: never commit (generate with ffmpeg at test time,
# see docs/contributing/development.md → Media fixtures)
internal/services/testdata/
+18
View File
@@ -85,6 +85,24 @@ were name-only across ALL libraries. Fixed; do not regress.
cleanups** — sync_integration's `DELETE FROM reading_history WHERE
true` destroyed the live reader's stats data on every suite run
until scoped to the test-sync users (cascade via user delete).
- **RUN ONLY YOUR OWN TESTS while iterating**: integration tests
require the containerized dev DB — a bare host `go test
./cmd/server/tests` fails on auth on every machine not running the
compose stack. Use the filtered container runner
(`docker compose -f docker-compose.yml -f
docker-compose.dev.yml --profile tests run --rm tests go test
./cmd/server/tests -run 'TestYourPrefix' -v`) and save full-suite
runs for pre-commit validation; several tests read live shared
state (system_settings, real library folders, the periodic backup
scan's websocket broadcasts) and flap on data/timing your change
never touched. Full recipes: docs/contributing/development.md
→ Testing.
- **NEVER COMMIT BINARY MEDIA** (mp3/m4b/flac/... fixtures):
`internal/services/testdata/` is gitignored by policy; generate
media at test runtime with ffmpeg instead (test images ship it).
ffmpeg cannot write Audible freeform atoms — those paths are unit-
tested with synthetic ffprobeReport values. Recipes:
docs/contributing/development.md → Media fixtures.
## Timezones
+4 -1
View File
@@ -59,7 +59,10 @@ RUN cp -r /go/pkg/mod /app/go-module-cache
# This stage is ONLY used for running tests, never deployed to production
FROM golang:1.26-alpine AS test-runner
RUN apk --no-cache add ca-certificates curl poppler-utils
# ffmpeg/ffprobe match the runtime stage: the audio ingestion (tag
# fallbacks, embedded covers, cue probing) must be exercised in tests
# exactly as it runs in production.
RUN apk --no-cache add ca-certificates curl poppler-utils ffmpeg
WORKDIR /app
+31
View File
@@ -0,0 +1,31 @@
info:
name: Get Audio Chapters
type: http
seq: 30
http:
method: GET
url: "{{base_url}}/api/media-items/{{media_item_id}}/chapters"
headers:
- name: ""
value: application/json
auth: inherit
settings:
encodeUrl: true
timeout: 0
followRedirects: true
maxRedirects: 5
docs: |-
## Get Audio Chapters
The format-abstraction chapter list for audiobooks
(media_items.audio_metadata). Single-file books return `layout: ""` and
an empty `tracks` array; folder-of-tracks books return `layout:
"folder"` and zip books `layout: "zip"`, each with one `tracks` entry
per playable file (global-seconds start/end plus the per-track `url`).
**Method:** GET
**Endpoint:** /api/media-items/{id}/chapters
+31
View File
@@ -0,0 +1,31 @@
info:
name: Get Audio Track
type: http
seq: 31
http:
method: GET
url: "{{base_url}}/api/media-items/{{media_item_id}}/audio/0"
headers:
- name: Range
value: bytes=0-1023
auth: inherit
settings:
encodeUrl: true
timeout: 0
followRedirects: true
maxRedirects: 5
docs: |-
## Get Audio Track
Streams one track of a multi-file audiobook (folder or zip layout).
Range requests are honored: folder tracks are served from the real
file; zip tracks are extracted in place (backward seeks re-open the
entry). Requires the item's library to be visible to the requester
(403 otherwise). Single-file books have no track manifest — 404.
**Method:** GET
**Endpoint:** /api/media-items/{id}/audio/{trackIndex}
@@ -0,0 +1,571 @@
package main
// Multi-file audiobook integration tests: folder-of-tracks and zip-of-
// audio books must ingest as ONE item each (never per-track items), the
// /chapters manifest must expose the track list, the per-track streaming
// endpoint must serve bytes behind the library-visibility gate, and
// rescans must be idempotent while absorbing pre-grouping one-track
// strays. Live dev DB — all fixtures use test-named libraries under
// t.TempDir() folders and cascade away with the library delete.
import (
"archive/zip"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"os/exec"
"path/filepath"
"testing"
"time"
"bookhoard/internal/database"
"github.com/google/uuid"
"github.com/jackc/pgx/v5/pgtype"
"github.com/stretchr/testify/require"
)
// writeTestTrack renders a short silent mp3 via ffmpeg when available so
// ffprobe can produce real durations; otherwise falls back to dummy bytes
// (ingest is ffprobe-optional by design — structure assertions don't need
// real audio).
func writeTestTrack(t *testing.T, path string) {
t.Helper()
require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o755))
if ffmpeg, err := exec.LookPath("ffmpeg"); err == nil {
cmd := exec.Command(ffmpeg, "-y", "-v", "error", "-f", "lavfi",
"-i", "sine=frequency=440:duration=0.3", "-codec:a", "libmp3lame", "-b:a", "32k", path)
if out, err := cmd.CombinedOutput(); err == nil {
return
} else {
t.Logf("ffmpeg track render failed (%s), falling back to dummy bytes: %s", err, out)
}
}
require.NoError(t, os.WriteFile(path, []byte("ID3 dummy audio"), 0o644))
}
// audioTestLibrary builds an audiobooks-type library with a fresh temp
// folder, returning its UUID and the folder path.
func audioTestLibrary(t *testing.T, setup *TestServerSetup, name string) (pgtype.UUID, string) {
t.Helper()
libID, folder := createTypedTestLibrary(t, setup, name, "audiobooks")
return libID, folder
}
// listLibraryAudioItems returns every media item row of the library.
func listLibraryAudioItems(t *testing.T, setup *TestServerSetup, libID pgtype.UUID) []database.ListMediaItemsByLibraryIncludingArchivedRow {
t.Helper()
items, err := setup.DB.ListMediaItemsByLibraryIncludingArchived(context.Background(), libID)
require.NoError(t, err)
return items
}
// findAudioItemByPath returns the item with the given file_path.
func findAudioItemByPath(t *testing.T, setup *TestServerSetup, libID pgtype.UUID, filePath string) (database.ListMediaItemsByLibraryIncludingArchivedRow, bool) {
t.Helper()
for _, item := range listLibraryAudioItems(t, setup, libID) {
if item.FilePath == filePath {
return item, true
}
}
return database.ListMediaItemsByLibraryIncludingArchivedRow{}, false
}
// decodeAudioPayload parses the audio_metadata JSONB of an item.
func decodeAudioPayload(t *testing.T, item database.ListMediaItemsByLibraryIncludingArchivedRow) (layout string, trackCount, chapterCount int, duration float64) {
t.Helper()
require.NotNil(t, item.AudioMetadata, "audio_metadata must be present")
var payload struct {
Layout string `json:"layout"`
DurationSeconds float64 `json:"duration_seconds"`
Tracks []struct {
Index int `json:"index"`
Path string `json:"path"`
} `json:"tracks"`
Chapters []json.RawMessage `json:"chapters"`
}
require.NoError(t, json.Unmarshal(item.AudioMetadata, &payload))
return payload.Layout, len(payload.Tracks), len(payload.Chapters), payload.DurationSeconds
}
func TestAudiobookFolderOfTracks(t *testing.T) {
setup := setupTestServer(t)
t.Cleanup(func() { _ = setup.Close() })
admin, _ := moveTestAdminID(t, setup)
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Folder %d", time.Now().UnixNano()))
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
// Author/Title/01.mp3, 02.mp3 — the canonical folder book.
writeTestTrack(t, filepath.Join(folder, "Test Author", "Test Title", "02 Second.mp3"))
writeTestTrack(t, filepath.Join(folder, "Test Author", "Test Title", "01 First.mp3"))
// A lone single-file book in a sibling folder must stay single-file.
writeTestTrack(t, filepath.Join(folder, "Test Author", "Lone Book", "lonely.mp3"))
scanTestFolder(t, setup, admin, folder)
// The folder is ONE item whose file_path is the directory.
book, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Test Author/Test Title"))
require.True(t, found, "folder book item must exist at the directory path")
layout, tracks, chapters, duration := decodeAudioPayload(t, book)
require.Equal(t, "folder", layout)
require.Equal(t, 2, tracks)
require.Equal(t, 2, chapters)
if duration <= 0 {
t.Log("no ffprobe durations available in this environment (degraded ingest)")
}
// The lone mp3 keeps the single-file model.
_, loneFound := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Test Author/Lone Book/lonely.mp3"))
require.True(t, loneFound, "lone mp3 must stay its own single-file item")
require.Len(t, listLibraryAudioItems(t, setup, libID), 2, "exactly two items: one folder book + one single-file book")
// Rescan idempotency: nothing duplicates, manifest unchanged.
scanTestFolder(t, setup, admin, folder)
require.Len(t, listLibraryAudioItems(t, setup, libID), 2)
again, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Test Author/Test Title"))
require.True(t, found)
require.JSONEq(t, string(book.AudioMetadata), string(again.AudioMetadata))
}
func TestAudiobookAbsorbsOneTrackStrays(t *testing.T) {
setup := setupTestServer(t)
t.Cleanup(func() { _ = setup.Close() })
admin, _ := moveTestAdminID(t, setup)
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Merge %d", time.Now().UnixNano()))
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
// First scan: a single mp3 → single-file item (the pre-grouping model).
writeTestTrack(t, filepath.Join(folder, "Merge Book", "01 only.mp3"))
scanTestFolder(t, setup, admin, folder)
_, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Merge Book/01 only.mp3"))
require.True(t, found, "single track ingests as its own item first")
// Second track arrives; rescan must create the folder book and delete
// the stray one-track item.
writeTestTrack(t, filepath.Join(folder, "Merge Book", "02 and.mp3"))
scanTestFolder(t, setup, admin, folder)
book, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Merge Book"))
require.True(t, found, "folder book must exist after rescan")
layout, tracks, _, _ := decodeAudioPayload(t, book)
require.Equal(t, "folder", layout)
require.Equal(t, 2, tracks)
items := listLibraryAudioItems(t, setup, libID)
require.Len(t, items, 1, "the one-track stray must be absorbed into the folder book")
}
func TestAudiobookZipBook(t *testing.T) {
setup := setupTestServer(t)
t.Cleanup(func() { _ = setup.Close() })
admin, _ := moveTestAdminID(t, setup)
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Zip %d", time.Now().UnixNano()))
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
// Build a zip of mp3s via ffmpeg-rendered tracks.
dir := t.TempDir()
writeTestTrack(t, filepath.Join(dir, "a.mp3"))
writeTestTrack(t, filepath.Join(dir, "b.mp3"))
zipPath := filepath.Join(folder, "Zipped Book.zip")
zipFiles(t, zipPath, map[string]string{
"01 first.mp3": filepath.Join(dir, "a.mp3"),
"02 second.mp3": filepath.Join(dir, "b.mp3"),
})
scanTestFolder(t, setup, admin, folder)
book, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Zipped Book.zip"))
require.True(t, found, "zip book item must exist at the zip path")
layout, tracks, chapters, _ := decodeAudioPayload(t, book)
require.Equal(t, "zip", layout)
require.Equal(t, 2, tracks)
require.Equal(t, 2, chapters)
// Rescan idempotency.
scanTestFolder(t, setup, admin, folder)
require.Len(t, listLibraryAudioItems(t, setup, libID), 1)
}
func TestAudiobookTrackEndpoints(t *testing.T) {
setup := setupTestServer(t)
t.Cleanup(func() { _ = setup.Close() })
admin, _ := moveTestAdminID(t, setup)
regularID := getRegularUserID(t, setup.DB)
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Serve %d", time.Now().UnixNano()))
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
writeTestTrack(t, filepath.Join(folder, "Serve Book", "01 one.mp3"))
writeTestTrack(t, filepath.Join(folder, "Serve Book", "02 two.mp3"))
scanTestFolder(t, setup, admin, folder)
book, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Serve Book"))
require.True(t, found)
bookID := uuid.UUID(book.ID.Bytes)
client := &http.Client{}
t.Run("chapters manifest exposes tracks", func(t *testing.T) {
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/"+bookID.String()+"/chapters", nil)
req.Header.Set("Authorization", "Bearer "+setup.Token)
resp, err := client.Do(req)
require.NoError(t, err)
defer func() { _ = resp.Body.Close() }()
require.Equal(t, http.StatusOK, resp.StatusCode)
var data struct {
Layout string `json:"layout"`
Tracks []struct {
Index int `json:"index"`
URL string `json:"url"`
Path string `json:"path"`
Start float64 `json:"start"`
} `json:"tracks"`
}
require.NoError(t, json.NewDecoder(resp.Body).Decode(&data))
require.Equal(t, "folder", data.Layout)
require.Len(t, data.Tracks, 2)
require.Equal(t, "/api/media-items/"+bookID.String()+"/audio/0", data.Tracks[0].URL)
})
t.Run("track streams with range support", func(t *testing.T) {
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/"+bookID.String()+"/audio/0", nil)
req.Header.Set("Authorization", "Bearer "+setup.Token)
req.Header.Set("Range", "bytes=0-99")
resp, err := client.Do(req)
require.NoError(t, err)
defer func() { _ = resp.Body.Close() }()
require.Equal(t, http.StatusPartialContent, resp.StatusCode)
})
t.Run("out-of-range track index is 404", func(t *testing.T) {
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/"+bookID.String()+"/audio/9", nil)
req.Header.Set("Authorization", "Bearer "+setup.Token)
resp, err := client.Do(req)
require.NoError(t, err)
defer func() { _ = resp.Body.Close() }()
require.Equal(t, http.StatusNotFound, resp.StatusCode)
})
t.Run("hidden library is 403 on chapters and tracks", func(t *testing.T) {
_, err := setup.DB.SetLibraryVisibility(context.Background(), database.SetLibraryVisibilityParams{
UserID: pgtype.UUID{Bytes: regularID, Valid: true},
LibraryID: libID,
IsVisible: false,
})
require.NoError(t, err)
for _, path := range []string{
"/api/media-items/" + bookID.String() + "/chapters",
"/api/media-items/" + bookID.String() + "/audio/0",
} {
req, _ := http.NewRequest("GET", setup.Server.URL+path, nil)
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
resp, err := client.Do(req)
require.NoError(t, err)
_ = resp.Body.Close()
require.Equal(t, http.StatusForbidden, resp.StatusCode, "%s must be gated", path)
}
// Restore visibility so harness cleanup ordering never sees a
// stale hidden row for a deleted library's user.
_, err = setup.DB.SetLibraryVisibility(context.Background(), database.SetLibraryVisibilityParams{
UserID: pgtype.UUID{Bytes: regularID, Valid: true},
LibraryID: libID,
IsVisible: true,
})
require.NoError(t, err)
})
}
// zipFiles builds a zip archive at zipPath mapping entry names to source
// file paths on disk.
func zipFiles(t *testing.T, zipPath string, entries map[string]string) {
t.Helper()
out, err := os.Create(zipPath)
require.NoError(t, err)
defer func() { _ = out.Close() }()
w := zip.NewWriter(out)
for name, src := range entries {
data, err := os.ReadFile(src)
require.NoError(t, err)
f, err := w.Create(name)
require.NoError(t, err)
_, err = f.Write(data)
require.NoError(t, err)
}
require.NoError(t, w.Close())
}
// fixtureAudio copies a committed binary fixture (real tagged media —
// the test container has no way to generate it) into a library folder.
// Path is relative from cmd/server/tests (three deep) to the services
// testdata tree.
func fixtureAudio(t *testing.T, name, dest string) {
t.Helper()
src := filepath.Join("..", "..", "..", "internal", "services", "testdata", "audio", name)
data, err := os.ReadFile(src)
require.NoError(t, err, "fixture %s must exist in the repo", name)
require.NoError(t, os.MkdirAll(filepath.Dir(dest), 0o755))
require.NoError(t, os.WriteFile(dest, data, 0o644))
}
// getJSONWithAuth fetches an API endpoint with the admin token and
// decodes the body into dst.
func getJSONWithAuth(t *testing.T, setup *TestServerSetup, path string, dst interface{}) int {
t.Helper()
req, _ := http.NewRequest("GET", setup.Server.URL+path, nil)
req.Header.Set("Authorization", "Bearer "+setup.Token)
resp, err := http.DefaultClient.Do(req)
require.NoError(t, err)
defer func() { _ = resp.Body.Close() }()
if dst != nil {
require.NoError(t, json.NewDecoder(resp.Body).Decode(dst))
}
return resp.StatusCode
}
func TestAudiobookNewContainerFormats(t *testing.T) {
setup := setupTestServer(t)
t.Cleanup(func() { _ = setup.Close() })
admin, _ := moveTestAdminID(t, setup)
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Formats %d", time.Now().UnixNano()))
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
for _, name := range []string{"book.opus", "book.ogg", "book.oga", "book.aac", "book.wav"} {
// one book per folder — a directory with ≥2 audio files is ONE
// book by the grouping doctrine
fixtureAudio(t, name, filepath.Join(folder, "Formats "+name, name))
}
scanTestFolder(t, setup, admin, folder)
items := listLibraryAudioItems(t, setup, libID)
require.Len(t, items, 5, "each new container ingests as its own single-file book")
mimes := map[string]string{}
for _, item := range items {
require.Equal(t, "audiobook", item.FormatGroup, "%s must classify as audiobook", item.FilePath)
mimes[filepath.Ext(item.FilePath)] = item.MimeType.String
}
require.Equal(t, map[string]string{
".opus": "audio/ogg", ".ogg": "audio/ogg", ".oga": "audio/ogg",
".aac": "audio/aac", ".wav": "audio/wav",
}, mimes)
// The download endpoint must serve the REAL content types — a wrong
// audio/mpeg fallback makes browsers refuse the stream.
for _, name := range []string{"book.opus", "book.aac", "book.wav"} {
item, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Formats "+name+"/"+name))
require.True(t, found, name)
bookID := uuid.UUID(item.ID.Bytes)
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/"+bookID.String()+"/download", nil)
req.Header.Set("Authorization", "Bearer "+setup.Token)
resp, err := http.DefaultClient.Do(req)
require.NoError(t, err)
// Drain but discard: a 0.3s sine is a few KB — nothing to save.
_, _ = io.Copy(io.Discard, resp.Body)
_ = resp.Body.Close()
require.Equal(t, http.StatusOK, resp.StatusCode)
require.Equal(t, item.MimeType.String, resp.Header.Get("Content-Type"), "%s content type", name)
}
}
func TestAudiobookTagFallbacks(t *testing.T) {
setup := setupTestServer(t)
t.Cleanup(func() { _ = setup.Close() })
admin, _ := moveTestAdminID(t, setup)
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Tags %d", time.Now().UnixNano()))
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
// No sidecars anywhere: the file's own tags are the only source.
fixtureAudio(t, "tagged.mp3", filepath.Join(folder, "Tag Author", "The Tagged Album", "tagged.mp3"))
scanTestFolder(t, setup, admin, folder)
item, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Tag Author/The Tagged Album/tagged.mp3"))
require.True(t, found)
require.Equal(t, "The Tagged Album", item.Title, "album tag names the book")
require.Equal(t, "Tag Author", item.Author.String, "album_artist tag names the author")
require.Len(t, item.Narrators, 1, "composer tag fills narrators (Audible convention)")
require.Equal(t, "Tag Narrator", item.Narrators[0])
require.Equal(t, "Tag Series", item.Series.String, "grouping tag fills series")
require.Equal(t, "Audiobook", item.Genre.String, "genre tag fills genre")
require.False(t, item.DatePublished.Time.IsZero(), "date tag fills publish date")
require.Equal(t, 2020, item.DatePublished.Time.Year())
// ASIN/abridged/subtitle arrive only through Audible freeform atoms,
// which ffmpeg cannot write into a fixture — those fallbacks are
// pinned by TestApplyAudioTagFallbacks (synthetic reports).
}
func TestAudiobookCueSheetChapters(t *testing.T) {
setup := setupTestServer(t)
t.Cleanup(func() { _ = setup.Close() })
admin, _ := moveTestAdminID(t, setup)
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Cue %d", time.Now().UnixNano()))
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
fixtureAudio(t, "cuebook.flac", filepath.Join(folder, "Cue Book", "cuebook.flac"))
fixtureAudio(t, "cuebook.cue", filepath.Join(folder, "Cue Book", "cuebook.cue"))
scanTestFolder(t, setup, admin, folder)
item, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Cue Book/cuebook.flac"))
require.True(t, found)
bookID := uuid.UUID(item.ID.Bytes)
var data struct {
Chapters []struct {
Index int `json:"index"`
Title string `json:"title"`
Start float64 `json:"start"`
End float64 `json:"end"`
} `json:"chapters"`
}
status := getJSONWithAuth(t, setup, "/api/media-items/"+bookID.String()+"/chapters", &data)
require.Equal(t, http.StatusOK, status)
require.Len(t, data.Chapters, 3, "cue tracks become chapters")
require.Equal(t, "Opening", data.Chapters[0].Title)
require.Equal(t, "The Middle Bit", data.Chapters[1].Title)
require.InDelta(t, 20.0, data.Chapters[1].Start, 0.01)
require.Equal(t, "Closing", data.Chapters[2].Title)
}
func TestAudiobookEmbeddedM4BChapters(t *testing.T) {
setup := setupTestServer(t)
t.Cleanup(func() { _ = setup.Close() })
admin, _ := moveTestAdminID(t, setup)
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio M4B %d", time.Now().UnixNano()))
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
fixtureAudio(t, "chaptered.m4b", filepath.Join(folder, "Chaptered Book.m4b"))
scanTestFolder(t, setup, admin, folder)
item, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Chaptered Book.m4b"))
require.True(t, found)
bookID := uuid.UUID(item.ID.Bytes)
var data struct {
Chapters []struct {
Title string `json:"title"`
} `json:"chapters"`
DurationSeconds float64 `json:"duration_seconds"`
}
status := getJSONWithAuth(t, setup, "/api/media-items/"+bookID.String()+"/chapters", &data)
require.Equal(t, http.StatusOK, status)
require.Len(t, data.Chapters, 2, "embedded m4b chapters parse")
require.Equal(t, "Part One", data.Chapters[0].Title)
require.Equal(t, "Part Two", data.Chapters[1].Title)
require.Greater(t, data.DurationSeconds, float64(0))
}
func TestAudiobookPlaylistOrdering(t *testing.T) {
setup := setupTestServer(t)
t.Cleanup(func() { _ = setup.Close() })
admin, _ := moveTestAdminID(t, setup)
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio M3U %d", time.Now().UnixNano()))
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
// Three tag-less tracks (natural order 01,02,10) plus a playlist
// imposing a DIFFERENT order and names.
fixtureAudio(t, "plain.mp3", filepath.Join(folder, "Playlist Book", "01_one.mp3"))
fixtureAudio(t, "plain.mp3", filepath.Join(folder, "Playlist Book", "02_two.mp3"))
fixtureAudio(t, "plain.mp3", filepath.Join(folder, "Playlist Book", "10_ten.mp3"))
playlist := "#EXTM3U\n#EXTINF:60,Epilogue\n./10_ten.mp3\n#EXTINF:60,Prologue\n01_one.mp3\n"
require.NoError(t, os.WriteFile(filepath.Join(folder, "Playlist Book", "book.m3u"), []byte(playlist), 0o644))
scanTestFolder(t, setup, admin, folder)
item, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Playlist Book"))
require.True(t, found, "folder book created")
require.NotNil(t, item.AudioMetadata)
var payload struct {
Tracks []struct {
Index int `json:"index"`
Path string `json:"path"`
Title string `json:"title"`
} `json:"tracks"`
}
require.NoError(t, json.Unmarshal(item.AudioMetadata, &payload))
require.Len(t, payload.Tracks, 3)
require.Equal(t, "Playlist Book/10_ten.mp3", payload.Tracks[0].Path, "playlist order beats natural sort")
require.Equal(t, "Epilogue", payload.Tracks[0].Title)
require.Equal(t, "Playlist Book/01_one.mp3", payload.Tracks[1].Path)
require.Equal(t, "Prologue", payload.Tracks[1].Title)
require.Equal(t, "Playlist Book/02_two.mp3", payload.Tracks[2].Path, "unlisted track appended in natural order")
}
func TestAudiobookEmbeddedCoverArt(t *testing.T) {
setup := setupTestServer(t)
t.Cleanup(func() { _ = setup.Close() })
admin, _ := moveTestAdminID(t, setup)
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Cover %d", time.Now().UnixNano()))
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
// Single file: embedded APIC → "<file>.cover.jpg" sibling.
fixtureAudio(t, "tagged.mp3", filepath.Join(folder, "Cover Book", "tagged.mp3"))
// Folder book: sidecar cover.jpg must WIN over the embedded art.
fixtureAudio(t, "tagged.mp3", filepath.Join(folder, "Sidecar Book", "01 track.mp3"))
fixtureAudio(t, "tagged.mp3", filepath.Join(folder, "Sidecar Book", "02 track.mp3"))
// A real (if tiny) sidecar cover next to the folder book:
fixtureAudio(t, "tagged.mp3", filepath.Join(folder, "Sidecar Book", "cover.jpg"))
scanTestFolder(t, setup, admin, folder)
single, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Cover Book/tagged.mp3"))
require.True(t, found)
if !ffmpegAvailableForTest() {
t.Skip("ffmpeg unavailable in this environment; embedded covers degrade gracefully")
}
require.NotEmpty(t, single.CoverImagePath.String, "embedded APIC becomes the cover")
coverRel := single.CoverImagePath.String
_, err := os.Stat(filepath.Join(folder, coverRel))
require.NoError(t, err, "cover sibling file exists on disk")
folder_, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Sidecar Book"))
require.True(t, found)
require.Equal(t, filepath.ToSlash("Sidecar Book/cover.jpg"), folder_.CoverImagePath.String,
"sidecar cover wins over embedded art")
}
func ffmpegAvailableForTest() bool {
_, err := os.Stat("/usr/bin/ffmpeg")
if err == nil {
return true
}
_, err = exec.LookPath("ffmpeg")
return err == nil
}
func TestRescanKeepsFilenameTitle(t *testing.T) {
// Regression pin: extractMetadata's default branch (.mobi/.azw/.pdb
// and unknown types) no longer pre-fills the filename as title, so
// the rescan path MUST apply the final filename fallback itself —
// a sidecar-less, tag-less book at a library root keeps its title
// instead of being blanked by a rescan.
setup := setupTestServer(t)
t.Cleanup(func() { _ = setup.Close() })
admin, _ := moveTestAdminID(t, setup)
libID, folder := createTypedTestLibrary(t, setup, fmt.Sprintf("Test Rescan Title %d", time.Now().UnixNano()), "ebooks")
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
// Library-root .mobi with no sidecars, no tags, no folder structure.
require.NoError(t, os.WriteFile(filepath.Join(folder, "Plain Book.mobi"), []byte("dummy mobi bytes"), 0o644))
scanTestFolder(t, setup, admin, folder)
item, found := findAudioItemByPath(t, setup, libID, "Plain Book.mobi")
require.True(t, found)
require.Equal(t, "Plain Book", item.Title, "create path falls back to filename")
bookID := uuid.UUID(item.ID.Bytes)
req, _ := http.NewRequest("POST", setup.Server.URL+"/api/media-items/"+bookID.String()+"/rescan", nil)
req.Header.Set("Authorization", "Bearer "+setup.Token)
resp, err := http.DefaultClient.Do(req)
require.NoError(t, err)
_ = resp.Body.Close()
require.Equal(t, http.StatusOK, resp.StatusCode)
rescanned, err := setup.DB.GetMediaItem(context.Background(), item.ID)
require.NoError(t, err)
require.Equal(t, "Plain Book", rescanned.Title, "rescan must not blank the filename-derived title")
}
+182
View File
@@ -0,0 +1,182 @@
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"os"
"path/filepath"
"testing"
"time"
"github.com/google/uuid"
"github.com/jackc/pgx/v5/pgtype"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// POST /api/scanner/cancel/:jobId is admin-only and must answer with
// distinct codes for unknown (404) and already-terminal (409) jobs.
func TestScanCancelEndpointSemantics(t *testing.T) {
setup := setupTestServer(t)
client := &http.Client{}
t.Run("cancel without admin role is forbidden", func(t *testing.T) {
url := fmt.Sprintf("%s/api/scanner/cancel/%s", setup.Server.URL, uuid.New().String())
req, _ := http.NewRequest("POST", url, nil)
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
resp, err := client.Do(req)
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
})
t.Run("cancel unknown job is not found", func(t *testing.T) {
url := fmt.Sprintf("%s/api/scanner/cancel/%s", setup.Server.URL, uuid.New().String())
req, _ := http.NewRequest("POST", url, nil)
req.Header.Set("Authorization", "Bearer "+setup.Token)
resp, err := client.Do(req)
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusNotFound, resp.StatusCode)
})
}
// Cancelling a manual scan mid-run must never corrupt library state:
// the abort skips the archive pass, so unvisited (but on-disk) files
// are not treated as missing. The canary item points at a file that is
// lexically LAST in the folder, so a mid-walk abort that (incorrectly)
// ran the archive pass would mark it missing and fail the assertion.
// The assertions hold under every interleaving — whether the cancel
// lands mid-scan or the scan already finished — so the test cannot
// flap against timing.
func TestScanCancelMidScanSafety(t *testing.T) {
setup := setupTestServer(t)
client := &http.Client{}
token := setup.Token
// Test library (helper registers API-delete cleanup so the run
// leaves no residue in the dev database).
libraryID := createTestLibraryWithFolder(t, setup.Server, token, "Test Scan Cancel Library", false)
libUUID, err := uuid.Parse(libraryID)
require.NoError(t, err)
// Folder on disk: many junk .epub files (zero bytes cannot parse as
// a zip/epub, so processMediaFile rejects each one and creates no
// rows) plus the canary file, lexically last.
dir := t.TempDir()
const fileCount = 1200
junk := bytes.Repeat([]byte{0}, 1024)
for i := 0; i < fileCount; i++ {
name := filepath.Join(dir, fmt.Sprintf("book-%04d.epub", i))
require.NoError(t, os.WriteFile(name, junk, 0o644))
}
canaryRel := "zzz-last.epub"
require.NoError(t, os.WriteFile(filepath.Join(dir, canaryRel), junk, 0o644))
// Register the folder with the library.
folderURL := fmt.Sprintf("%s/api/libraries/%s/folders", setup.Server.URL, libraryID)
body, _ := json.Marshal(map[string]interface{}{"folder_path": dir})
req, _ := http.NewRequest("POST", folderURL, bytes.NewBuffer(body))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer "+token)
resp, err := client.Do(req)
require.NoError(t, err)
_ = resp.Body.Close()
require.Equal(t, http.StatusCreated, resp.StatusCode, "folder creation should succeed")
// Pre-existing item whose file exists on disk. A normal completed
// scan never marks it missing; neither may a cancelled one.
_, err = setup.DBPool.Exec(context.Background(),
"INSERT INTO media_items (library_id, title, file_path) VALUES ($1, $2, $3)",
pgtype.UUID{Bytes: libUUID, Valid: true}, "Test Cancel Canary", canaryRel)
require.NoError(t, err)
// Start the scan and cancel it as soon as it is accepted.
jobID := startScan(t, client, setup.Server.URL, token, libraryID)
cancelURL := fmt.Sprintf("%s/api/scanner/cancel/%s", setup.Server.URL, jobID)
cancelReq, _ := http.NewRequest("POST", cancelURL, nil)
cancelReq.Header.Set("Authorization", "Bearer "+token)
cancelResp, err := client.Do(cancelReq)
require.NoError(t, err)
_ = cancelResp.Body.Close()
// 200 = cancelled mid-run; 409 = the scan already finished before
// the cancel landed. Both are acceptable; the assertions below hold
// either way.
assert.Contains(t, []int{http.StatusOK, http.StatusConflict}, cancelResp.StatusCode)
// Poll to a terminal state.
final := pollJobToTerminal(t, client, setup.Server.URL, token, jobID)
assert.Contains(t, []string{"completed", "cancelled"}, final["status"],
"scan must finish cleanly or be cancelled, got %v (error: %v)", final["status"], final["error"])
// The safety invariant: no item was marked missing or archived, and
// the junk files created no phantom rows.
var items, missingSum, archived int
err = setup.DBPool.QueryRow(context.Background(),
`SELECT COUNT(*),
COALESCE(SUM(missing_scan_count), 0),
COUNT(*) FILTER (WHERE archived_at IS NOT NULL)
FROM media_items WHERE library_id = $1`,
pgtype.UUID{Bytes: libUUID, Valid: true}).Scan(&items, &missingSum, &archived)
require.NoError(t, err)
assert.Equal(t, 1, items, "junk files must not create media items")
assert.Zero(t, missingSum, "cancelled/completed scan must not mark on-disk files missing")
assert.Zero(t, archived, "no item may be archived by this scan")
// Cancelling a terminal job is a conflict, whichever terminal state
// the job reached.
cancelReq2, _ := http.NewRequest("POST", cancelURL, nil)
cancelReq2.Header.Set("Authorization", "Bearer "+token)
cancelResp2, err := client.Do(cancelReq2)
require.NoError(t, err)
_ = cancelResp2.Body.Close()
assert.Equal(t, http.StatusConflict, cancelResp2.StatusCode)
}
func startScan(t *testing.T, client *http.Client, baseURL, token, libraryID string) string {
url := fmt.Sprintf("%s/api/libraries/%s/scan", baseURL, libraryID)
req, _ := http.NewRequest("POST", url, nil)
req.Header.Set("Authorization", "Bearer "+token)
resp, err := client.Do(req)
require.NoError(t, err)
defer resp.Body.Close()
require.Equal(t, http.StatusAccepted, resp.StatusCode)
var decoded map[string]interface{}
require.NoError(t, json.NewDecoder(resp.Body).Decode(&decoded))
jobID, ok := decoded["job_id"].(string)
require.True(t, ok, "job_id should be a string")
require.NotEmpty(t, jobID)
return jobID
}
func pollJobToTerminal(t *testing.T, client *http.Client, baseURL, token, jobID string) map[string]interface{} {
url := fmt.Sprintf("%s/api/scanner/status/%s", baseURL, jobID)
for i := 0; i < 30; i++ {
req, _ := http.NewRequest("GET", url, nil)
req.Header.Set("Authorization", "Bearer "+token)
resp, err := client.Do(req)
require.NoError(t, err)
var status map[string]interface{}
err = json.NewDecoder(resp.Body).Decode(&status)
_ = resp.Body.Close()
require.NoError(t, err)
switch status["status"] {
case "completed", "failed", "cancelled":
return status
}
time.Sleep(500 * time.Millisecond)
}
t.Fatalf("job %s did not reach a terminal state within 15s", jobID)
return nil
}
+6 -2
View File
@@ -244,8 +244,12 @@ CREATE TABLE IF NOT EXISTS media_items (
-- { "narrators": [], "duration_seconds": N, "codec_profile": "LC",
-- "bit_rate": N, "asin": "...", "chapters": [
-- { "index": 0, "title": "...", "start": 0.0, "end": 168.0 } ] }
-- The chapters array IS the format abstraction: embedded m4b chapters
-- and (later) folder-as-book files both write this exact shape.
-- The chapters array IS the format abstraction: embedded m4b chapters,
-- folder-of-tracks books and zip-of-audio books all write this exact
-- shape. Multi-file books add additive keys: "layout"
-- ("folder" | "zip"; absent = single file) and "tracks" (one entry
-- per playable file with global-seconds start/end plus the path the
-- /audio/:trackIndex endpoint resolves).
audio_metadata JSONB
);
+17
View File
@@ -102,3 +102,20 @@ cover_image_path, has_conflict, format_group, badge`.
origin — its rows are unchanged. Request-parse widening documented per
rule 2; response shapes untouched; locked by
TestOriginWriterAttribution.
## Changelog entry — multi-file audiobooks (2026-10-18)
- `GET /api/media-items/:id/chapters` gained ADDITIVE response keys
`layout` ("" for single-file books, `"folder"` or `"zip"` for
multi-file books) and `tracks[]` (`index, url, title, path, start,
end, duration_seconds, size_bytes`; empty for single-file books).
Existing keys (`media_item_id, chapters, duration_seconds,
codec_profile`) unchanged — no pin existed on this endpoint, so no
pin widened; recorded here per rule 2's spirit.
- New sibling endpoint `GET /api/media-items/:id/audio/:trackIndex`
streams one track of a multi-file book (Range-capable). New surface,
not a modification; gated by library visibility like `ServeFile`
(403 on hidden libraries).
- `media_items.audio_metadata` payload gained additive JSON keys
`layout` and `tracks` (see schema.sql comment). Consumers parse
additively; stored single-file payloads are unchanged.
+72 -7
View File
@@ -258,17 +258,82 @@ go tool cover -html=coverage.out
### Integration Tests
Integration tests hit the LIVE dev database and bind-mounted paths, both
of which exist only inside the Docker stack. A bare host `go test
./cmd/server/tests` fails with database-auth errors on every machine
that isn't running the compose stack — the container is the only
supported runner.
```bash
# Run integration tests (may hit rate limits)
go test ./cmd/server/tests -v
# Run the whole integration suite (pre-commit validation — NOT for
# iterating on a change; some tests are coupled to the shared dev DB's
# live state and this environment)
make test-integration
# Run with test mode (recommended)
TEST_MODE=true RATE_LIMIT_ENABLED=false go test ./cmd/server/tests -v
# Run specific test
go test ./cmd/server/tests -run TestAuth -v
# Iterate on YOUR change: run only your own tests, in the container
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile tests run --rm tests \
go test ./cmd/server/tests -run 'TestYourPrefix' -v -timeout 600s
```
**Run only your own tests while iterating.** A full-suite run rebuilds
the test image and takes minutes; more importantly, several tests read
live state (the dev DB's `system_settings`, real library folders, a
periodic backup scan broadcasting over websocket) and can fail on
timing or on data your change never touched. Filtered runs are cheap,
deterministic for your scope, and don't fight the live reader for the
database.
Test-created data rules (harness cleanup depends on them): users use
`@tests.bookhoard.internal` / `@example.com` emails, libraries and
collections carry "test"-prefixed names, and cleanups delete test
libraries (cascades handle the rest) rather than table-wide deletes.
### Media Fixtures (audio/books)
**Never commit binary media.** The Go toolchain and Docker prod builds
ignore `internal/services/testdata/` entirely (gitignored by policy —
see `.gitignore`), and the production image never copies `internal/`.
Generate what a test needs at runtime with ffmpeg (shipped in both the
test-runner and runtime images):
```bash
# The five audio containers (audiobook library accepts all of these)
ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a libmp3lame -b:a 32k plain.mp3
ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a flac book.flac
ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a libopus -b:a 24k book.opus
ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a libvorbis -q:a 1 book.ogg
ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a aac -b:a 32k book.aac
ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a pcm_s16le book.wav
# Tags (album=book title, album_artist=author, composer=narrator —
# Audible's own convention — plus disc/track ordering keys)
ffmpeg -i in.mp3 -codec copy \
-metadata title="Chapter Three" -metadata album="The Book" \
-metadata album_artist="Author" -metadata composer="Narrator" \
-metadata track="3/9" -metadata disc="2/3" \
-metadata grouping="Series" -metadata genre="Audiobook" -metadata date=2020 out.mp3
# Embedded cover art (mp3 APIC / m4b covr): the picture rides as the
# only video stream
ffmpeg -i cover.jpg -i audio-only.mp3 -map 1:a -map 0:v -c:v copy \
-disposition:v attached_pic tagged.mp3
# Embedded m4b chapters: an ffmetadata sidecar file
printf ';FFMETADATA1\n[CHAPTER]\nTIMEBASE=1/1000\nSTART=0\nEND=500\ntitle=Part One\n' > chapters.txt
ffmpeg -i in.m4b -i chapters.txt -map 0:a -map 1 -map_metadata 1 -codec copy -f ipod chaptered.m4b
```
CUE sheets and m3u playlists are plain text — write them inline in the
test. **The one thing ffmpeg cannot synthesize: Audible freeform atoms**
(`----:com.apple.iTunes:SERIES`, `:SUBTITLE`, `:ASIN`, `:ABRIDGED`) —
those ingestion paths are covered by unit tests using synthetic
`ffprobeReport` values instead of binary files.
Manual end-to-end testing against the live system: drop fixture books
into `uploads/Audiobooks/Ingestion Fixtures/` — **one book per
subfolder** (a directory with ≥2 audio files ingests as ONE
folder-of-tracks book) — and run a library scan from the admin UI.
### API Testing with Bruno OpenCollection YAML
```bash
@@ -0,0 +1,53 @@
# Get Audio Chapters
The format-abstraction chapter list for audiobooks, sourced from
`media_items.audio_metadata`. Embedded m4b chapters, folder-of-tracks
books and zip-of-audio books all write the same chapter shape; the
locator `{chapter, audio_offset_seconds}` and this endpoint never learn
which file model produced them.
**Endpoint**: `GET /api/media-items/{id}/chapters`
**Auth**: Required (JWT)
## Response
```json
{
"media_item_id": "…",
"chapters": [
{ "index": 0, "title": "Chapter One", "start": 0.0, "end": 168.0 }
],
"duration_seconds": 36000.0,
"codec_profile": "mp3",
"layout": "folder",
"tracks": [
{
"index": 0,
"url": "/api/media-items/{id}/audio/0",
"title": "01 First",
"path": "Author/Title/01 First.mp3",
"start": 0.0,
"end": 168.0,
"duration_seconds": 168.0,
"size_bytes": 2700000
}
]
}
```
- `layout` is `""` for single-file books (the Batch A model — the whole
file streams through the `/uploads` path), `"folder"` for a directory
of track files and `"zip"` for a zip-of-audio book.
- `tracks` is empty for single-file books and carries one entry per
playable file otherwise. `start`/`end` are cumulative GLOBAL seconds —
the same coordinate system chapters, bookmarks and
`audio_offset_seconds` use.
## Error Responses
| Code | Description |
| ---- | ------------------------------- |
| 400 | Invalid media item id |
| 401 | Missing/invalid token |
| 404 | Book not found |
| 500 | audio_metadata failed to parse |
@@ -0,0 +1,44 @@
# Get Audio Track
Streams one playable track of a multi-file audiobook (folder-of-tracks
or zip-of-audio book). Track indexes and bounds come from the track
manifest on [Get Audio Chapters](get_audio_chapters.md).
**Endpoint**: `GET /api/media-items/{id}/audio/{trackIndex}`
**Auth**: Required (JWT — Bearer header or session cookie; the web
player's `<audio>` element authenticates by cookie)
## Serving
- **Folder books**: the track is served straight from its real file
(`http.ServeContent`) — full `Range` support.
- **Zip books**: the entry is extracted in place. Forward seeks discard
bytes in the open stream; backward seeks re-open the entry and skip
(deflate has no random access — cheap at audio bitrates). `Range`
requests are honored.
Single-file books have no track manifest — they stream through the
`/uploads` path exactly as before and this endpoint answers 404.
## Library visibility
The item's library must be visible to the requester; a hidden library
answers **403** on every track (same gate as the download endpoint).
## Error Responses
| Code | Description |
| ---- | -------------------------------------------- |
| 400 | Invalid media item id or track index |
| 401 | Missing/invalid token |
| 403 | Library not visible to the requester |
| 404 | Book/track not found, or single-file book |
| 500 | audio_metadata failed to parse |
## Example
```bash
curl -H "Authorization: Bearer $TOKEN" \
-H "Range: bytes=0-1023" \
"http://localhost:8765/api/media-items/550e8400-.../audio/0"
```
@@ -90,8 +90,11 @@ Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
### Absent data
`Audio Tracks` (Audiobookshelf's per-file count) is deliberately absent:
Bookhoard audiobooks are one file per book until multi-file support
lands, so a file count would just equal `items`. See HANDOFF item 9.
multi-file audiobooks (folder-of-tracks / zip books) now ingest as ONE
item whose track list lives in `audio_metadata.tracks`, so a file count
would still not equal a per-book "tracks" metric without a new query.
Add it by aggregating `jsonb_array_length(audio_metadata->'tracks')` if
the panel is ever wanted.
## Error Responses
+83
View File
@@ -0,0 +1,83 @@
# Audiobook metadata: sources & precedence
Every audiobook ingestion path (single-file, folder-of-tracks, zip-of-
audio) fills the same fields from the same ranked sources. Each source
**only fills fields the earlier ones left empty** — curated data always
outranks guesses — and every source is optional: a book still ingests
(title from the filename) when all of them are absent. Admin edits made
after ingest are protected by the metadata-override mechanism on
rescans, regardless of source quality.
## Book-level metadata (title, author, narrators, series, …)
| Rank | Source | Feeds |
|---|---|---|
| 1 | **Sidecars** — `metadata.opf` (Calibre), `metadata.json` (Audiobookshelf/Libation, array-or-string tolerant) | title (+subtitle join), author, series + index, narrators, abridged, ASIN, tags/genres, dates, publisher, description |
| 2 | **Folder structure** — `Author/Title`, `Author/Series, Book #N - Title`, `Author/Series/Title` | title, author, series + index |
| 3 | **Embedded tags** (via ffprobe) | `album` → title; `album_artist` → author (`artist` fallback); `composer` → **narrators** (Audible's own convention); `grouping` → series; `date` → publish date; `genre` → genre; Audible freeform atoms `----:com.apple.iTunes:SERIES`/`:SERIES-PART`/`:SUBTITLE`/`:ASIN`/`:ABRIDGED` |
| 4 | **Filename/folder name** | title last of all (created only — never lets a rescan blank a title) |
Abridged is the one guarded field: an explicit sidecar `false` wins even
when a tag says `true` (the sidecar's presence is tracked, not just its
value). Narrators from the composer tag stay ONE string — composer
values have no reliable multi-narrator delimiter, and splitting
"Vance, Simon" would manufacture fake people.
## Chapters (the `{index, title, start, end}` array)
| Rank | Source |
|---|---|
| 1 | Sidecar `metadata.json` `chapters[]` (curated, global seconds) |
| 2 | Embedded m4b chapter atoms (`chpl` / QuickTime text track, read by ffprobe) |
| 3 | **CUE sheet** — `.cue` next to the audio file (FILE-directive matched, `INDEX 01 MM:SS:FF` at 75 fps); the classic one-big-flac + cue rip shape |
| 4 | Track boundaries — for folder/zip books, each file is a chapter |
## Track ordering (multi-file books)
| Rank | Source |
|---|---|
| 1 | **Playlist** — exactly one `.m3u`/`.m3u8` among the book root's direct children; its entry order wins and `#EXTINF` names the tracks |
| 2 | **Tags** — `(disc, track)` sort when *every* track carries a track number (fixes flat `01.mp3` dumps from multi-disc rips) |
| 3 | **Natural sort** of relative paths (`02` before `10`), recursive — the default |
Track display titles, highest first: `movement_name` → `title` tag →
playlist `#EXTINF` → cleaned filename (leading ordinals/separators
stripped).
## Book grouping (which files form one book)
- The book root is the nearest ancestor holding audio files DIRECTLY,
merged upward through disc-part directories (`CD1`, `Disc 2`, `Part 3`,
`05`) when the parent has no audio of its own.
- Sibling directories are NEVER merged (two books under one author stay
two books); a root with a single track falls back to the single-file
model.
- A lone `.zip` of audio files is its own book (layout "zip"); a folder
of tracks is one book with `file_path` = the directory (layout
"folder"); a single audio file is the Batch A model (no layout key).
- Rescans take a fast path (track count + total size + known durations
unchanged → no re-probe) and absorb pre-grouping one-track strays.
## Covers
| Rank | Source |
|---|---|
| 1 | Sidecar `cover.jpg` / `folder.jpg` / … next to the book |
| 2 | **Embedded art** (`APIC` mp3, `covr` m4b, flac picture) — one optional ffmpeg call, written as a `<book>.cover.jpg` sibling following the EPUB convention |
## Accepted containers (audiobook libraries)
`.m4b` `.m4a` `.mp3` `.flac` `.zip` (book container) `.opus` `.ogg`
`.oga` `.aac` `.wav` — all browser-playable; served with their real
media types (`audio/ogg` per RFC 7845 for Opus-in-Ogg, etc.).
## Documented non-goals
- **ID3v2 CHAP/CTOC chapter frames** — the written MP3 standard for
embedded chapters, near-zero audiobook adoption; would need a
hand-rolled ID3v2.4 parser. Revisit only if real files show up.
- **NFO sidecars** — no genuine audiobook NFO standard exists (Jellyfin's
Bookshelf uses Calibre OPF, which we read).
- **ReplayGain / EBU R128 loudness tags** — a playback feature, not
ingestion; separate plan if ever wanted.
- **`.wma`** — browsers cannot play it; excluded deliberately.
+5
View File
@@ -574,6 +574,11 @@ type Querier interface {
UpdateMediaBookmarkForSync(ctx context.Context, arg UpdateMediaBookmarkForSyncParams) (MediaBookmarks, error)
UpdateMediaHighlightForSync(ctx context.Context, arg UpdateMediaHighlightForSyncParams) (MediaHighlights, error)
UpdateMediaItem(ctx context.Context, arg UpdateMediaItemParams) (MediaItems, error)
// Focused refresh for multi-file audiobooks (folder-of-tracks / zip
// books): rescans recompute the track manifest and the aggregate facts
// that follow from it. Editorial fields stay untouched — admins may have
// edited title/narrators after ingest.
UpdateMediaItemAudioBookPayload(ctx context.Context, arg UpdateMediaItemAudioBookPayloadParams) (MediaItems, error)
UpdateMediaItemChapterMetadata(ctx context.Context, arg UpdateMediaItemChapterMetadataParams) (MediaItems, error)
// Update media item format
UpdateMediaItemFormat(ctx context.Context, arg UpdateMediaItemFormatParams) (MediaItemFormats, error)
+105
View File
@@ -14297,6 +14297,111 @@ func (q *Queries) UpdateMediaItem(ctx context.Context, arg UpdateMediaItemParams
return i, err
}
const UpdateMediaItemAudioBookPayload = `-- name: UpdateMediaItemAudioBookPayload :one
UPDATE media_items
SET file_size = $2,
mime_type = $3,
audio_metadata = $4,
chapter_count = $5,
updated_at = NOW()
WHERE id = $1
RETURNING id, library_id, title, author, isbn, description, file_path, file_size, mime_type, cover_image_path, series, series_number, tags, narrators, abridged, asin, date_published, publisher, contributors, language, edition, page_count, genre, copyright_year, goodreads_id, openlibrary_id, google_books_id, added_by_admin_id, created_at, imported_at, updated_at, format_group, format_mimetype, is_reflowable, has_fixed_layout, total_characters, chapter_count, entitlement_id, revision_number, kobo_content_id, kobo_metadata, manga_type, reading_direction, series_count, volume, imprint, age_rating, web_url, story_arc, is_black_and_white, metadata_notes, community_rating, alternate_info, scan_information, summary, metadata_overrides, missing_scan_count, archived_at, chapter_metadata, library_type_name, audio_metadata, tags_search, contributors_search, narrators_search, file_sha256, opf_identifier, opf_uuid, hash_confidence
`
type UpdateMediaItemAudioBookPayloadParams struct {
ID pgtype.UUID `db:"id" json:"id"`
FileSize pgtype.Int8 `db:"file_size" json:"file_size"`
MimeType pgtype.Text `db:"mime_type" json:"mime_type"`
AudioMetadata []byte `db:"audio_metadata" json:"audio_metadata"`
ChapterCount pgtype.Int4 `db:"chapter_count" json:"chapter_count"`
}
// Focused refresh for multi-file audiobooks (folder-of-tracks / zip
// books): rescans recompute the track manifest and the aggregate facts
// that follow from it. Editorial fields stay untouched — admins may have
// edited title/narrators after ingest.
func (q *Queries) UpdateMediaItemAudioBookPayload(ctx context.Context, arg UpdateMediaItemAudioBookPayloadParams) (MediaItems, error) {
row := q.db.QueryRow(ctx, UpdateMediaItemAudioBookPayload,
arg.ID,
arg.FileSize,
arg.MimeType,
arg.AudioMetadata,
arg.ChapterCount,
)
var i MediaItems
err := row.Scan(
&i.ID,
&i.LibraryID,
&i.Title,
&i.Author,
&i.Isbn,
&i.Description,
&i.FilePath,
&i.FileSize,
&i.MimeType,
&i.CoverImagePath,
&i.Series,
&i.SeriesNumber,
&i.Tags,
&i.Narrators,
&i.Abridged,
&i.Asin,
&i.DatePublished,
&i.Publisher,
&i.Contributors,
&i.Language,
&i.Edition,
&i.PageCount,
&i.Genre,
&i.CopyrightYear,
&i.GoodreadsID,
&i.OpenlibraryID,
&i.GoogleBooksID,
&i.AddedByAdminID,
&i.CreatedAt,
&i.ImportedAt,
&i.UpdatedAt,
&i.FormatGroup,
&i.FormatMimetype,
&i.IsReflowable,
&i.HasFixedLayout,
&i.TotalCharacters,
&i.ChapterCount,
&i.EntitlementID,
&i.RevisionNumber,
&i.KoboContentID,
&i.KoboMetadata,
&i.MangaType,
&i.ReadingDirection,
&i.SeriesCount,
&i.Volume,
&i.Imprint,
&i.AgeRating,
&i.WebUrl,
&i.StoryArc,
&i.IsBlackAndWhite,
&i.MetadataNotes,
&i.CommunityRating,
&i.AlternateInfo,
&i.ScanInformation,
&i.Summary,
&i.MetadataOverrides,
&i.MissingScanCount,
&i.ArchivedAt,
&i.ChapterMetadata,
&i.LibraryTypeName,
&i.AudioMetadata,
&i.TagsSearch,
&i.ContributorsSearch,
&i.NarratorsSearch,
&i.FileSha256,
&i.OpfIdentifier,
&i.OpfUuid,
&i.HashConfidence,
)
return i, err
}
const UpdateMediaItemChapterMetadata = `-- name: UpdateMediaItemChapterMetadata :one
UPDATE media_items
SET chapter_metadata = $2, updated_at = NOW()
+14
View File
@@ -3229,3 +3229,17 @@ WHERE mi.archived_at IS NULL
AND ($2::uuid[] IS NULL OR mi.library_id = ANY($2::uuid[]))
ORDER BY mi.file_size DESC, mi.title ASC
LIMIT 5;
-- name: UpdateMediaItemAudioBookPayload :one
-- Focused refresh for multi-file audiobooks (folder-of-tracks / zip
-- books): rescans recompute the track manifest and the aggregate facts
-- that follow from it. Editorial fields stay untouched — admins may have
-- edited title/narrators after ingest.
UPDATE media_items
SET file_size = $2,
mime_type = $3,
audio_metadata = $4,
chapter_count = $5,
updated_at = NOW()
WHERE id = $1
RETURNING id, library_id, title, author, isbn, description, file_path, file_size, mime_type, cover_image_path, series, series_number, tags, narrators, abridged, asin, date_published, publisher, contributors, language, edition, page_count, genre, copyright_year, goodreads_id, openlibrary_id, google_books_id, added_by_admin_id, created_at, imported_at, updated_at, format_group, format_mimetype, is_reflowable, has_fixed_layout, total_characters, chapter_count, entitlement_id, revision_number, kobo_content_id, kobo_metadata, manga_type, reading_direction, series_count, volume, imprint, age_rating, web_url, story_arc, is_black_and_white, metadata_notes, community_rating, alternate_info, scan_information, summary, metadata_overrides, missing_scan_count, archived_at, chapter_metadata, library_type_name, audio_metadata, tags_search, contributors_search, narrators_search, file_sha256, opf_identifier, opf_uuid, hash_confidence;
+231 -4
View File
@@ -1,6 +1,7 @@
package handlers
import (
"archive/zip"
"bookhoard/internal/database"
"bookhoard/internal/services"
wsync "bookhoard/internal/sync"
@@ -2550,9 +2551,16 @@ func (mh *MediaHandler) ServeFile(c *echo.Context) error {
}
// Check if file exists
if _, err := os.Stat(fullPath); os.IsNotExist(err) {
fullInfo, err := os.Stat(fullPath)
if err != nil {
return c.JSON(http.StatusNotFound, map[string]string{"error": "file not found"})
}
// Folder-of-tracks audiobooks store their book root DIRECTORY in
// file_path; there is no single file to serve and ServeFile must not
// fall through to directory listing behavior.
if fullInfo.IsDir() {
return c.JSON(http.StatusNotFound, map[string]string{"error": "multi-file books have no single download"})
}
ext := strings.ToLower(filepath.Ext(fullPath))
contentType := services.MimeTypes[ext]
@@ -2648,9 +2656,12 @@ func (mh *MediaHandler) saveCoverImage(c echo.Context, mediaUUID uuid.UUID, file
// GetAudioChapters handles GET /api/media-items/:id/chapters — the
// format-abstraction chapter list for audiobooks (media_items
// .audio_metadata). Embedded m4b chapters and folder-as-book files write
// the same shape. Empty list when the item isn't audio or hasn't been
// probed.
// .audio_metadata). Embedded m4b chapters, folder-of-tracks books and zip
// books write the same shape. Multi-file books additionally carry
// `layout` and a `tracks` manifest whose URLs point at the per-track
// streaming endpoint; single-file books keep the empty layout their
// existing consumers expect. Empty list when the item isn't audio or
// hasn't been probed.
func (mh *MediaHandler) GetAudioChapters(c *echo.Context) error {
mediaID := c.Param("id")
mediaUUID, err := uuid.Parse(mediaID)
@@ -2662,6 +2673,20 @@ func (mh *MediaHandler) GetAudioChapters(c *echo.Context) error {
if err != nil {
return c.JSON(http.StatusNotFound, map[string]string{"error": "book not found"})
}
if !item.LibraryID.Valid {
return c.JSON(http.StatusNotFound, map[string]string{"error": "book has no library"})
}
// Library visibility gate (family-lens rule): chapter lists are media
// metadata — a hidden library must not leak them any more than its
// file bytes.
visibleSet, _, err := mh.visibleLibrarySet(c)
if err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to check library access"})
}
if !visibleSet[uuid.UUID(item.LibraryID.Bytes).String()] {
return c.JSON(http.StatusForbidden, map[string]string{"error": "library not accessible"})
}
var payload services.AudioMetadataPayload
if len(item.AudioMetadata) > 0 {
@@ -2670,10 +2695,212 @@ func (mh *MediaHandler) GetAudioChapters(c *echo.Context) error {
}
}
tracks := make([]map[string]interface{}, 0, len(payload.Tracks))
for _, tr := range payload.Tracks {
tracks = append(tracks, map[string]interface{}{
"index": tr.Index,
"url": fmt.Sprintf("/api/media-items/%s/audio/%d", mediaID, tr.Index),
"title": tr.Title,
"path": tr.Path,
"start": tr.Start,
"end": tr.End,
"duration_seconds": tr.DurationSeconds,
"size_bytes": tr.SizeBytes,
})
}
return c.JSON(http.StatusOK, map[string]interface{}{
"media_item_id": item.ID,
"chapters": payload.Chapters,
"duration_seconds": payload.DurationSeconds,
"codec_profile": payload.CodecProfile,
"layout": payload.Layout,
"tracks": tracks,
})
}
// GetAudioTrack handles GET /api/media-items/:id/audio/:trackIndex — the
// per-track byte stream for multi-file audiobooks (folder books serve the
// real file; zip books extract the entry in place). Library visibility is
// the permission gate, same as ServeFile.
func (mh *MediaHandler) GetAudioTrack(c *echo.Context) error {
mediaUUID, err := uuid.Parse(c.Param("id"))
if err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid media item id"})
}
trackIndex, err := strconv.Atoi(c.Param("trackIndex"))
if err != nil || trackIndex < 0 {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid track index"})
}
item, err := mh.db.GetMediaItem(c.Request().Context(), pgtype.UUID{Bytes: mediaUUID, Valid: true})
if err != nil {
return c.JSON(http.StatusNotFound, map[string]string{"error": "book not found"})
}
if !item.LibraryID.Valid {
return c.JSON(http.StatusNotFound, map[string]string{"error": "book has no library"})
}
// Library visibility gate (family-lens rule): a hidden library means
// its audiobooks' bytes are not reachable either.
visibleSet, _, err := mh.visibleLibrarySet(c)
if err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to check library access"})
}
if !visibleSet[uuid.UUID(item.LibraryID.Bytes).String()] {
return c.JSON(http.StatusForbidden, map[string]string{"error": "library not accessible"})
}
var payload services.AudioMetadataPayload
if len(item.AudioMetadata) > 0 {
if err := json.Unmarshal(item.AudioMetadata, &payload); err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to parse audio metadata"})
}
}
if trackIndex >= len(payload.Tracks) {
return c.JSON(http.StatusNotFound, map[string]string{"error": "track not found"})
}
track := payload.Tracks[trackIndex]
switch payload.Layout {
case "folder":
fullPath, err := mh.getFullFilePath(c.Request().Context(), item.LibraryID, track.Path)
if err != nil {
return c.JSON(http.StatusNotFound, map[string]string{"error": "track file not found"})
}
f, err := os.Open(fullPath)
if err != nil {
return c.JSON(http.StatusNotFound, map[string]string{"error": "track file not found"})
}
defer func() { _ = f.Close() }()
info, err := f.Stat()
if err != nil || info.IsDir() {
return c.JSON(http.StatusNotFound, map[string]string{"error": "track file not found"})
}
c.Response().Header().Set("Content-Type", audioTrackContentType(track.Path))
c.Response().Header().Set("Cache-Control", "private, max-age=86400")
// http.ServeContent wants a ReadSeeker for Range support; the os
// file provides both.
http.ServeContent(c.Response(), c.Request(), filepath.Base(fullPath), info.ModTime(), f)
return nil
case "zip":
zipPath, err := mh.getFullFilePath(c.Request().Context(), item.LibraryID, item.FilePath)
if err != nil {
return c.JSON(http.StatusNotFound, map[string]string{"error": "archive not found"})
}
seeker, err := newZipEntrySeeker(zipPath, track.Path)
if err != nil {
return c.JSON(http.StatusNotFound, map[string]string{"error": "track not found in archive"})
}
defer func() { _ = seeker.Close() }()
c.Response().Header().Set("Content-Type", audioTrackContentType(track.Path))
c.Response().Header().Set("Cache-Control", "private, max-age=86400")
// Zip entries re-open+skip for backward seeks (deflate has no
// random access), which audio bitrates make cheap enough.
http.ServeContent(c.Response(), c.Request(), filepath.Base(track.Path), time.Time{}, seeker)
return nil
default:
// Single-file books stream through the /uploads path; the track
// endpoint has nothing to slice.
return c.JSON(http.StatusNotFound, map[string]string{"error": "book has no track manifest"})
}
}
// audioTrackContentType maps a track path/entry name to its mime type.
func audioTrackContentType(name string) string {
if m := services.MimeTypes[strings.ToLower(filepath.Ext(name))]; m != "" {
return m
}
return "audio/mpeg"
}
// zipEntrySeeker is an io.ReadSeeker over one zip entry. Forward seeks
// discard bytes in the open stream; backward seeks re-open the entry and
// skip — the only option over a deflate stream, and cheap at audio
// bitrates. Owns the archive reader for the life of the request.
type zipEntrySeeker struct {
zrc *zip.ReadCloser
rc io.ReadCloser
pos int64
size int64
entryFn func() (io.ReadCloser, error)
}
func newZipEntrySeeker(zipPath, entryName string) (*zipEntrySeeker, error) {
zrc, err := zip.OpenReader(zipPath)
if err != nil {
return nil, err
}
var entry *zip.File
for _, f := range zrc.File {
if filepath.ToSlash(f.Name) == filepath.ToSlash(entryName) {
entry = f
break
}
}
if entry == nil {
_ = zrc.Close()
return nil, fmt.Errorf("entry %q not found", entryName)
}
open := func() (io.ReadCloser, error) { return entry.Open() }
rc, err := open()
if err != nil {
_ = zrc.Close()
return nil, err
}
return &zipEntrySeeker{
zrc: zrc,
rc: rc,
size: int64(entry.UncompressedSize64),
entryFn: open,
}, nil
}
func (z *zipEntrySeeker) Read(p []byte) (int, error) {
n, err := z.rc.Read(p)
z.pos += int64(n)
return n, err
}
func (z *zipEntrySeeker) Seek(offset int64, whence int) (int64, error) {
var target int64
switch whence {
case io.SeekStart:
target = offset
case io.SeekCurrent:
target = z.pos + offset
case io.SeekEnd:
target = z.size + offset
default:
return z.pos, fmt.Errorf("invalid whence %d", whence)
}
if target < 0 {
return z.pos, fmt.Errorf("negative seek target %d", target)
}
if target < z.pos {
// Deflate streams can't rewind: restart the entry and skip.
if err := z.rc.Close(); err != nil {
return z.pos, err
}
rc, err := z.entryFn()
if err != nil {
return z.pos, err
}
z.rc = rc
z.pos = 0
}
if remaining := target - z.pos; remaining > 0 {
if _, err := io.CopyN(io.Discard, z.rc, remaining); err != nil {
return z.pos, err
}
z.pos = target
}
return z.pos, nil
}
func (z *zipEntrySeeker) Close() error {
_ = z.rc.Close()
return z.zrc.Close()
}
+9 -6
View File
@@ -916,9 +916,16 @@ func (h *OPDSHandler) DownloadBook(c *echo.Context) error {
}
// Check if file exists
if _, err := os.Stat(filePath); os.IsNotExist(err) {
fileInfo, err := os.Stat(filePath)
if err != nil {
return c.JSON(http.StatusNotFound, map[string]string{"error": "file not found"})
}
// Folder-of-tracks audiobooks point at a book root DIRECTORY: there
// is no single file to stream for OPDS clients (a directory fd reads
// EISDIR). Zip books are real files and keep working.
if fileInfo.IsDir() {
return c.JSON(http.StatusNotFound, map[string]string{"error": "multi-file books have no single download"})
}
// Open file
file, err := os.Open(filePath)
@@ -927,11 +934,7 @@ func (h *OPDSHandler) DownloadBook(c *echo.Context) error {
}
defer file.Close()
// Get file info
fileInfo, err := file.Stat()
if err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to get file info"})
}
// Get file info (fileInfo already statted above)
// Set headers
filename := filepath.Base(filePath)
+23
View File
@@ -3,6 +3,7 @@ package handlers
import (
"bookhoard/internal/services"
"context"
"errors"
"fmt"
"net/http"
@@ -209,6 +210,28 @@ func (h *Handler) GetScanSummary(c *echo.Context) error {
})
}
// CancelScan handles POST /api/scanner/cancel/:jobId (admin): requests
// cancellation of a pending or running scan job. Cancellation is
// cooperative — the file currently being processed finishes, the walk
// stops at its next context check, and the archive pass is skipped so
// unvisited files are never treated as missing.
func (h *Handler) CancelScan(c *echo.Context) error {
jobID := c.Param("jobId")
if err := h.worker.CancelJob(jobID); err != nil {
if errors.Is(err, services.ErrJobNotFound) {
return c.JSON(http.StatusNotFound, map[string]string{"error": "job not found"})
}
return c.JSON(http.StatusConflict, map[string]string{"error": err.Error()})
}
return c.JSON(http.StatusOK, map[string]interface{}{
"message": "scan cancellation requested",
"job_id": jobID,
"status": "cancelled",
})
}
// StartWatchModeForLibrary starts watching a specific library's folders
func (h *Handler) StartWatchModeForLibrary(ctx context.Context, libraryID pgtype.UUID, adminID pgtype.UUID) error {
h.mu.Lock()
+4 -3
View File
@@ -18,10 +18,11 @@ func registerMediaRoutes(cfg *Config) {
// Book download endpoint - same ServeFile flow as /uploads/library-:id/*
// (JWT + library-visibility gated), addressed by media item ID.
protected.GET("/media-items/:id/download", cfg.MediaHandler.ServeFile)
// The format-abstraction chapter list (audio_metadata). Streaming
// itself rides the existing /uploads Range-serving path with cookie
// auth — no separate endpoint.
// The format-abstraction chapter list (audio_metadata). Single-file
// books stream through the existing /uploads Range-serving path;
// multi-file books additionally expose per-track streaming.
protected.GET("/media-items/:id/chapters", cfg.MediaHandler.GetAudioChapters)
protected.GET("/media-items/:id/audio/:trackIndex", cfg.MediaHandler.GetAudioTrack)
// Media rating routes (all authenticated users)
protected.POST("/media-items/:id/rating", cfg.MediaHandler.CreateMediaRating)
+5
View File
@@ -217,6 +217,11 @@ func RegisterRoutes(cfg *Config) *handlers.Handler {
mime.AddExtensionType(".m4a", "audio/mp4")
mime.AddExtensionType(".mp3", "audio/mpeg")
mime.AddExtensionType(".flac", "audio/flac")
mime.AddExtensionType(".opus", "audio/ogg")
mime.AddExtensionType(".ogg", "audio/ogg")
mime.AddExtensionType(".oga", "audio/ogg")
mime.AddExtensionType(".aac", "audio/aac")
mime.AddExtensionType(".wav", "audio/wav")
e := cfg.Echo
// Set up validator
+1
View File
@@ -17,6 +17,7 @@ func registerScannerRoutes(protected, admin *echo.Group, h *handlers.Handler) {
admin.POST("/scanner/scan", h.ScanLibrary)
admin.POST("/scanner/start", h.StartScanner)
admin.POST("/scanner/stop", h.StopScanner)
admin.POST("/scanner/cancel/:jobId", h.CancelScan)
admin.GET("/scanner/status/:jobId", h.GetScanStatus)
admin.GET("/scanner/active", h.GetActiveScans)
+14 -1
View File
@@ -38,7 +38,11 @@ var AllowedExtensions = map[string][]string{
// All four decided breadth formats registered from day one (the
// DOC/LIT doctrine: extensions are a deliberate list); only m4b
// scans richly in Batch A — the chapter abstraction carries the rest.
LibraryTypeAudiobooks: {".m4b", ".m4a", ".mp3", ".flac"},
// .zip joins as the zip-of-audio book container (multi-file ingest);
// comics libraries keep using .cbz so the two never collide. The
// browser-playable lossless/next-gen containers join for Batch B
// metadata parity (.wma deliberately absent: browsers can't play it).
LibraryTypeAudiobooks: {".m4b", ".m4a", ".mp3", ".flac", ".zip", ".opus", ".ogg", ".oga", ".aac", ".wav"},
}
var MimeTypes = map[string]string{
@@ -74,6 +78,15 @@ var MimeTypes = map[string]string{
".m4a": "audio/mp4",
".mp3": "audio/mpeg",
".flac": "audio/flac",
// Opus is registered as audio/ogg per RFC 7845 (Ogg encapsulation);
// every browser <audio> element accepts it that way.
".opus": "audio/ogg",
".ogg": "audio/ogg",
".oga": "audio/ogg",
".aac": "audio/aac",
".wav": "audio/wav",
// Zip-of-audio book container (audiobooks libraries only).
".zip": "application/zip",
}
// GetLibraryTypes retrieves all available library types
+166 -35
View File
@@ -140,6 +140,12 @@ type MediaScanner struct {
newItems int
errors int
job *Job
// audioBookRoots tracks the folder-book roots (absolute paths)
// already ingested during THIS scan walk, so per-track walk visits
// skip instead of reprocessing. Reset at the top of ScanFolders; the
// archive pass reuses it to keep book roots off the missing list.
audioBookRoots map[string]bool
}
// CalibreOPFMetadata represents intermediate parsed metadata from Calibre metadata.opf files
@@ -419,6 +425,14 @@ func (s *MediaScanner) enqueueLibraryScan(rootFolder string) {
}
func (s *MediaScanner) ScanFolders(ctx context.Context) error {
// Cancellation is checked at every stage boundary and inside the
// walk callback: the in-flight file finishes, then the scan aborts
// and reports context.Canceled (processJob classifies that as a
// cancelled job, not a failed one).
if ctx.Err() != nil {
return ctx.Err()
}
if len(s.folders) == 0 {
return fmt.Errorf("no folders set")
}
@@ -426,8 +440,12 @@ func (s *MediaScanner) ScanFolders(ctx context.Context) error {
s.totalFiles = 0
s.newItems = 0
s.errors = 0
s.audioBookRoots = make(map[string]bool)
for _, folder := range s.folders {
if ctx.Err() != nil {
return ctx.Err()
}
if err := filepath.WalkDir(folder, func(path string, d fs.DirEntry, err error) error {
if !d.IsDir() && s.isScannableFile(path) {
s.totalFiles++
@@ -445,6 +463,10 @@ func (s *MediaScanner) ScanFolders(ctx context.Context) error {
mediaFiles := 0
for _, folder := range s.folders {
if ctx.Err() != nil {
return ctx.Err()
}
fmt.Printf("Scanning folder: %s\n", folder)
if _, err := os.Stat(folder); os.IsNotExist(err) {
@@ -454,6 +476,10 @@ func (s *MediaScanner) ScanFolders(ctx context.Context) error {
}
err := filepath.WalkDir(folder, func(path string, d fs.DirEntry, err error) error {
if ctx.Err() != nil {
return fs.SkipAll
}
if err != nil {
fmt.Printf("Error accessing path %s: %v\n", path, err)
s.errors++
@@ -480,6 +506,19 @@ func (s *MediaScanner) ScanFolders(ctx context.Context) error {
}
}
// Multi-file audiobooks first: zip-of-audio books and
// folder-of-tracks books ingest as whole units, and tracks
// inside an already-ingested root skip entirely. Everything
// else (lone audio files, all non-audio media) falls
// through to the single-file path below.
if handled, err := s.maybeProcessAudioBook(ctx, path); handled {
if err != nil {
fmt.Printf("Error processing audio book %s: %v\n", path, err)
s.errors++
}
return nil
}
wasNew, err := s.processMediaFile(ctx, path)
if err != nil {
fmt.Printf("Error processing media file %s: %v\n", path, err)
@@ -502,6 +541,14 @@ func (s *MediaScanner) ScanFolders(ctx context.Context) error {
fmt.Printf("Scan completed: %d total files scanned, %d media files found, %d new items, %d errors\n",
processedFiles, mediaFiles, s.newItems, s.errors)
// A cancelled scan must not run the archive lifecycle pass below:
// it never visited every folder, so unvisited files would be
// treated as missing and archived (and eventually purged). Abort
// instead — cancelling never destroys library data.
if ctx.Err() != nil {
return ctx.Err()
}
// Archive lifecycle pass. Items whose files vanished from disk are
// archived after two consecutive missing scans (reading history kept,
// item hidden), and purged for good once archived older than the
@@ -509,6 +556,11 @@ func (s *MediaScanner) ScanFolders(ctx context.Context) error {
// Every branch logs - the previous hard-delete cleanup failed silently
// and left orphaned rows undetected.
for _, folder := range s.folders {
if ctx.Err() != nil {
fmt.Printf("[ARCHIVE] Scan cancelled, stopping archive pass\n")
return ctx.Err()
}
lib, err := s.db.GetLibraryByFolder(ctx, folder)
if err != nil {
fmt.Printf("[ARCHIVE] Warning: no library found for folder %s, skipping archive pass: %v\n", folder, err)
@@ -537,6 +589,14 @@ func (s *MediaScanner) ScanFolders(ctx context.Context) error {
continue // Skip to next folder to avoid false archivals
}
// Folder-book roots count as present: the item's file_path is the
// DIRECTORY, which the file walk above never emits.
for root := range s.audioBookRoots {
if strings.HasPrefix(root, folder) {
scannedPaths[s.getRelativePath(root)] = true
}
}
for _, item := range dbItems {
if item.FilePath == "" || scannedPaths[item.FilePath] {
continue // File present; unarchive is handled in processMediaFile
@@ -795,16 +855,6 @@ func (s *MediaScanner) processMediaFile(ctx context.Context, path string) (bool,
metadata = &MediaMetadata{}
}
// Audiobooks: sidecar narrators/chapters + ffprobe facts into the
// audio_metadata JSONB (sidecar-first, ffprobe-fills; the payload's
// shape is the chapter abstraction every format writes).
if isAudioFile(path) {
if metadata.AudioMeta == nil {
metadata.AudioMeta = readSidecarAudio(path)
}
s.enrichAudioMetadata(path, metadata)
}
// Extract hash information during metadata extraction
hashInfo, formatInfo, err := s.extractHashInfo(path)
if err != nil {
@@ -921,6 +971,21 @@ func (s *MediaScanner) processMediaFile(ctx context.Context, path string) (bool,
}
}
// Audiobooks: sidecar narrators/chapters + ffprobe facts (incl. the
// full tag fallbacks) into the audio_metadata JSONB. Runs AFTER the
// folder-structure pass so the precedence chain holds: sidecars →
// folder structure → tags → filename. The payload's shape is the
// chapter abstraction every format writes.
if isAudioFile(path) {
if metadata.AudioMeta == nil {
metadata.AudioMeta = readSidecarAudio(path)
}
s.enrichAudioMetadata(path, metadata)
if metadata.CoverPath == "" {
s.applyEmbeddedCoverFallback(path, metadata)
}
}
// Final fallback if still missing essential metadata
if metadata.Title == "" {
metadata.Title = strings.TrimSuffix(filepath.Base(path), filepath.Ext(path))
@@ -956,6 +1021,7 @@ func (s *MediaScanner) processMediaFile(ctx context.Context, path string) (bool,
SeriesNumber: pgtype.Int4{Int32: metadata.SeriesNumber, Valid: metadata.SeriesNumber > 0},
Publisher: pgtype.Text{String: metadata.Publisher, Valid: metadata.Publisher != ""},
DatePublished: pgtype.Date{Time: metadata.PublishDate, Valid: !metadata.PublishDate.IsZero()},
Genre: pgtype.Text{String: metadata.Genre, Valid: metadata.Genre != ""},
Contributors: metadata.Contributors,
ContributorsSearch: contributorsSearch,
Narrators: nilToEmpty(metadata.Narrators),
@@ -1086,6 +1152,16 @@ func (s *MediaScanner) applyEmbeddedCoverFallback(path string, metadata *MediaMe
if coverPath, err := s.extractEPUBCover(path); err == nil && coverPath != "" {
metadata.CoverPath = s.getRelativePath(coverPath)
}
default:
// Audio containers (mp3 APIC, m4b/flac embedded picture): one
// ffmpeg call, written as a "<file>.cover.jpg" sibling — the
// same convention the EPUB path uses. Purely optional: absent
// ffmpeg or no embedded art simply leaves the cover empty.
if isAudioFile(path) && !strings.EqualFold(filepath.Ext(path), ".zip") {
if cover := extractEmbeddedAudioCover(path, path); cover != "" {
metadata.CoverPath = s.getRelativePath(cover)
}
}
}
}
@@ -1680,10 +1756,10 @@ func (s *MediaScanner) extractMetadata(path string) (*MediaMetadata, error) {
}
return metadata, nil
default:
// For other formats, return basic metadata
return &MediaMetadata{
Title: strings.TrimSuffix(filepath.Base(path), ext),
}, nil
// Audio containers and other unknown types: NO filename title
// here — processMediaFile's final fallback applies the filename
// LAST, after the audio tag-fallback chain has had its turn.
return &MediaMetadata{}, nil
}
}
@@ -3161,20 +3237,29 @@ func countArchiveImages(filePath string) (int, error) {
// from the existing row so rescans never clobber user customizations; only
// reset-to-scanned-defaults (RescanMediaItem with reset) clears them.
func (s *MediaScanner) updateMediaItem(ctx context.Context, existing database.MediaItems, path string) error {
// Re-extract metadata for the update
metadata, err := s.extractMetadata(path)
if err != nil {
fmt.Printf("Warning: failed to extract metadata for force rescan %s: %v\n", path, err)
metadata = &MediaMetadata{}
}
return s.updateMediaItemWithMetadata(ctx, existing, path, nil)
}
// Rescans must RE-DERIVE audio facts, never preserve stale ones
// (gotcha 39's principle applied to audio_metadata).
if isAudioFile(path) {
if metadata.AudioMeta == nil {
metadata.AudioMeta = readSidecarAudio(path)
// updateMediaItemWithMetadata is updateMediaItem with an optional
// pre-extracted metadata set. Multi-file audiobooks arrive through
// RescanMediaItem with book-level metadata (sidecars, folder structure,
// first-track tags) because the generic per-file extraction would read
// the book ROOT/CONTAINER as if it were the book file.
func (s *MediaScanner) updateMediaItemWithMetadata(ctx context.Context, existing database.MediaItems, path string, preExtracted *MediaMetadata) error {
var metadata *MediaMetadata
if preExtracted != nil {
metadata = preExtracted
} else {
// Re-extract metadata for the update
extracted, err := s.extractMetadata(path)
if err != nil {
fmt.Printf("Warning: failed to extract metadata for force rescan %s: %v\n", path, err)
metadata = &MediaMetadata{}
} else if extracted == nil {
metadata = &MediaMetadata{}
} else {
metadata = extracted
}
s.enrichAudioMetadata(path, metadata)
}
// Folder-structure fallback, mirroring the CREATE path
@@ -3201,6 +3286,35 @@ func (s *MediaScanner) updateMediaItem(ctx context.Context, existing database.Me
}
}
// Audiobooks (single-file): re-derive the audio facts + tag fallbacks
// AFTER the folder-structure pass — the precedence chain (sidecars →
// folder structure → tags → filename) must hold on rescans too.
// Multi-file books arrive with preExtracted metadata already carrying
// their enriched payload.
// Rescans must RE-DERIVE audio facts, never preserve stale ones
// (gotcha 39's principle applied to audio_metadata).
if preExtracted == nil && isAudioFile(path) {
if metadata.AudioMeta == nil {
metadata.AudioMeta = readSidecarAudio(path)
}
s.enrichAudioMetadata(path, metadata)
if metadata.CoverPath == "" {
s.applyEmbeddedCoverFallback(path, metadata)
}
}
// Final filename fallback, mirroring processMediaFile's create path:
// extractMetadata's default branch no longer pre-fills the filename
// as title (embedded tags must outrank it), so the update path needs
// the same last rung — a sidecar-less, tag-less book at a library
// root (no folder structure to match) keeps its filename-derived
// title instead of being blanked. Guarded to single-file updates:
// multi-file representatives are sentinels/containers whose names
// must never leak as titles.
if preExtracted == nil && metadata.Title == "" {
metadata.Title = strings.TrimSuffix(filepath.Base(path), filepath.Ext(path))
}
// Normalize metadata fields
metadata.Contributors = utils.NormalizeContributors(metadata.Contributors)
metadata.Tags = utils.NormalizeTags(metadata.Tags)
@@ -3257,21 +3371,26 @@ func (s *MediaScanner) updateMediaItem(ctx context.Context, existing database.Me
utils.ApplyMetadataOverrides(&params, existing)
params.MetadataOverrides = utils.MergeOverrides(existing.MetadataOverrides)
if _, err = s.db.UpdateMediaItem(ctx, params); err != nil {
if _, err := s.db.UpdateMediaItem(ctx, params); err != nil {
return err
}
// Format flags are file-derived facts, not user metadata: refresh them on
// every update so a force rescan (the admin's backfill tool) can
// reclassify rows created before a classifyFormatGroup arm existed.
// Mirrors the creation path above.
ext := strings.ToLower(filepath.Ext(path))
var epubIsFixedLayout bool
if ext == ".epub" {
isFixed, fixedErr := s.DetectFixedLayoutEPUB(path)
epubIsFixedLayout = fixedErr == nil && isFixed
// Mirrors the creation path above. Multi-file audiobooks keep the
// "audiobook" group: their representative path is a directory or a zip
// container, which no classify arm could name.
formatGroup, isReflowable, hasFixedLayout := "audiobook", false, false
if preExtracted == nil || preExtracted.AudioMeta == nil || preExtracted.AudioMeta.Layout == "" {
ext := strings.ToLower(filepath.Ext(path))
var epubIsFixedLayout bool
if ext == ".epub" {
isFixed, fixedErr := s.DetectFixedLayoutEPUB(path)
epubIsFixedLayout = fixedErr == nil && isFixed
}
formatGroup, isReflowable, hasFixedLayout = classifyFormatGroup(ext, epubIsFixedLayout)
}
formatGroup, isReflowable, hasFixedLayout := classifyFormatGroup(ext, epubIsFixedLayout)
mimeType := s.getMimeType(path)
if err := s.db.UpdateMediaItemFormatGroup(ctx, database.UpdateMediaItemFormatGroupParams{
ID: existing.ID,
@@ -3335,6 +3454,16 @@ func (s *MediaScanner) RescanMediaItem(ctx context.Context, mediaItemID pgtype.U
return fmt.Errorf("failed to stat media file: %w", err)
}
// Multi-file audiobooks (folder/zip layout) must NOT go through the
// generic per-file updater: it would read the book ROOT directory (or
// the zip container) as if it were the book file — deriving a filename
// title and classifying format_group "unknown", which drops the item
// out of the audio reader. The book-level path rebuilds the track
// manifest and re-derives metadata from sidecars/tags instead.
if item.FormatGroup == "audiobook" && audioLayoutOf(item.AudioMetadata) != "" {
return s.rescanMultiFileAudioBook(ctx, item, fullPath)
}
if err := s.updateMediaItem(ctx, item, fullPath); err != nil {
return fmt.Errorf("failed to update media item: %w", err)
}
@@ -3980,6 +4109,8 @@ func (s *MediaScanner) detectFormatType(filePath string) string {
return "pdf"
case ".cbz", ".cbr", ".cb7", ".cbt":
return "comic_archive"
case ".m4b", ".m4a", ".mp3", ".flac", ".opus", ".ogg", ".oga", ".aac", ".wav":
return "audiobook"
case ".mobi":
return "mobi"
case ".azw", ".azw3":
@@ -4010,7 +4141,7 @@ func classifyFormatGroup(ext string, epubIsFixedLayout bool) (formatGroup string
return "fixed_layout", false, true
case ".cbz", ".cbr", ".cb7", ".cbt":
return "comic_archive", false, true
case ".m4b", ".m4a", ".mp3", ".flac":
case ".m4b", ".m4a", ".mp3", ".flac", ".opus", ".ogg", ".oga", ".aac", ".wav":
// Audio is neither reflowable text nor fixed-layout pages; the
// "audiobook" group routes readers to the audio player instead
// of any document engine.
+222 -5
View File
@@ -9,10 +9,17 @@ package services
// until a rescan runs with ffprobe present).
//
// The chapters array written into media_items.audio_metadata IS the
// format abstraction: embedded m4b chapters and (later) folder-as-book
// files both produce {index, title, start, end} entries, so the locator
// {chapter, offset_seconds} and the /chapters endpoint never learn which
// file model produced them.
// format abstraction: embedded m4b chapters, folder-as-book files and
// zip-of-audio books all produce {index, title, start, end} entries, so
// the locator {chapter, offset_seconds} and the /chapters endpoint never
// learn which file model produced them.
//
// Multi-file books (layout "folder" | "zip", see
// media_scanner_audio_multifile.go) additionally carry a tracks array:
// one entry per playable file, with its global-seconds start/end and the
// path the /audio/:trackIndex streaming endpoint resolves. Single-file
// books leave layout and tracks empty — the <audio> element streams the
// whole file exactly as before.
import (
"encoding/json"
@@ -33,6 +40,21 @@ type AudioChapter struct {
End float64 `json:"end"`
}
// AudioTrack is one playable file of a multi-file audiobook. Path is the
// library-relative file path (layout "folder") or the in-zip entry name
// (layout "zip"); the /audio/:trackIndex endpoint resolves it to bytes.
// Start/End are cumulative global seconds across the book — the same
// coordinate system chapters, bookmarks and audio_offset_seconds use.
type AudioTrack struct {
Index int `json:"index"`
Path string `json:"path"`
Title string `json:"title,omitempty"`
Start float64 `json:"start"`
End float64 `json:"end"`
DurationSeconds float64 `json:"duration_seconds,omitempty"`
SizeBytes int64 `json:"size_bytes,omitempty"`
}
// AudioMetadataPayload is the JSONB stored in media_items.audio_metadata.
// Only the audio paths read it — ebook paths never do (the kobo_metadata
// precedent).
@@ -44,16 +66,33 @@ type AudioMetadataPayload struct {
ASIN string `json:"asin,omitempty"`
Abridged *bool `json:"abridged,omitempty"`
Chapters []AudioChapter `json:"chapters,omitempty"`
// Layout: "" = single file (the Batch A model), "folder" = a
// directory of track files, "zip" = a zip archive of tracks.
Layout string `json:"layout,omitempty"`
// Tracks: one entry per playable file for multi-file layouts. Empty
// for single-file books (the whole file IS the track).
Tracks []AudioTrack `json:"tracks,omitempty"`
}
func isAudioFile(path string) bool {
switch strings.ToLower(filepath.Ext(path)) {
case ".m4b", ".m4a", ".mp3", ".flac":
case ".m4b", ".m4a", ".mp3", ".flac", ".zip", ".opus", ".ogg", ".oga", ".aac", ".wav":
return true
}
return false
}
// isAudioTrackFile reports whether a path is a PLAYABLE track of a
// multi-file book. Zips are book CONTAINERS (layout "zip"), never tracks
// of a folder book — a nested zip inside a book root must not join the
// track list, and pre-grouping one-track strays were never zips.
func isAudioTrackFile(path string) bool {
if strings.EqualFold(filepath.Ext(path), ".zip") {
return false
}
return isAudioFile(path)
}
// ffprobeAvailable is resolved once per scan process; a missing binary is
// a degradation, not an error.
var ffprobeAvailable = func() bool {
@@ -61,10 +100,55 @@ var ffprobeAvailable = func() bool {
return err == nil
}()
// ffmpegAvailable mirrors ffprobeAvailable for the embedded-cover
// extractor (ffprobe cannot write image streams; ffmpeg can).
var ffmpegAvailable = func() bool {
_, err := exec.LookPath("ffmpeg")
return err == nil
}()
// extractEmbeddedAudioCover pulls the embedded cover art (mp3 APIC /
// m4b covr / flac picture) out of an audio file with one ffmpeg call —
// the attached picture presents itself as the file's only video stream.
// Writes outputBase+".cover.jpg" following the EPUB sidecar convention
// ("<book>.cover.jpg"). Returns "" when ffmpeg is absent, the file
// carries no picture, or anything fails — purely optional enrichment.
func extractEmbeddedAudioCover(inputPath, outputBase string) string {
if !ffmpegAvailable {
return ""
}
outPath := outputBase + ".cover.jpg"
cmd := exec.Command("ffmpeg",
"-y", "-v", "error",
"-i", inputPath,
"-map", "0:v:0",
"-frames:v", "1",
outPath,
)
if out, err := cmd.CombinedOutput(); err != nil {
// "matches no streams" is the no-embedded-art case; stay quiet
// about it, log anything unexpected.
if len(out) > 0 && !strings.Contains(string(out), "matches no streams") {
fmt.Printf("ffmpeg cover extraction note for %s: %s\n", inputPath, strings.TrimSpace(string(out)))
}
return ""
}
if info, err := os.Stat(outPath); err != nil || info.Size() == 0 {
_ = os.Remove(outPath)
return ""
}
return outPath
}
type ffprobeReport struct {
Format struct {
Duration flexFloat `json:"duration"`
BitRate flexFloat `json:"bit_rate"`
// Tags: ffprobe normalizes well-known atoms to lowercase keys but
// passes freeform keys (Audible's `----:com.apple.iTunes:*`)
// through verbatim — a map with case-insensitive lookup is the
// only honest shape. See tag().
Tags map[string]string `json:"tags"`
} `json:"format"`
Streams []struct {
CodecName string `json:"codec_name"`
@@ -134,6 +218,24 @@ func runFFProbe(path string) *ffprobeReport {
return &report
}
// tag returns the first non-empty ffprobe tag among the candidate keys,
// matched case-insensitively and trimmed. Empty string when none match.
func (r *ffprobeReport) tag(keys ...string) string {
if r == nil {
return ""
}
for _, key := range keys {
for k, v := range r.Format.Tags {
if strings.EqualFold(k, key) {
if trimmed := strings.TrimSpace(v); trimmed != "" {
return trimmed
}
}
}
}
return ""
}
// enrichAudioMetadata merges ffprobe facts into the (possibly sidecar-
// seeded) audio payload, creating it when no sidecar existed. Sidecar
// narrators/chapters win (curated); ffprobe fills duration, codec,
@@ -192,6 +294,24 @@ func (s *MediaScanner) enrichAudioMetadata(path string, metadata *MediaMetadata)
if payload.Abridged != nil {
metadata.Abridged = *payload.Abridged
}
// Tag fallbacks run LAST in the single-file chain (sidecars → folder
// structure handled by the caller → tags → filename): they only fill
// fields the earlier sources left empty.
applyAudioTagFallbacks(metadata, report)
// CUE sheets are the last chapter source: sidecar chapters and
// embedded container chapters (m4b) outrank them.
if len(payload.Chapters) == 0 {
if cueChapters := readCueChapters(path); len(cueChapters) > 0 {
payload.Chapters = cueChapters
if payload.DurationSeconds == 0 {
if last := cueChapters[len(cueChapters)-1]; last.End > 0 {
payload.DurationSeconds = last.End
}
}
}
}
}
// parseSidecarAudioFields extracts the audio-only fields from a
@@ -261,6 +381,103 @@ func marshalAudioMetadata(payload *AudioMetadataPayload) []byte {
return out
}
// applyAudioTagFallbacks fills EMPTY media metadata from the file's own
// tags — the last rung of the ingestion precedence chain (curated
// sidecars → folder structure → embedded tags → filename). Field map:
//
// album / album_artist / artist → title / author
// composer → narrators (Audible's own
// convention: the narrator lives in the composer tag)
// grouping → series
// ----:com.apple.iTunes:SERIES[-PART] → series + index (Libation
// preserves these Audible freeform atoms)
// ----:com.apple.iTunes:SUBTITLE → joined into the title
// ("Title: Subtitle"), mirroring the ABS sidecar path
// ----:com.apple.iTunes:ASIN / ABRIDGED → asin / abridged flag
// date → publish date; genre → genre
//
// Abridged is the one field where an explicit sidecar false must not be
// overridden by a tag — the payload pointer marks sidecar presence.
func applyAudioTagFallbacks(metadata *MediaMetadata, report *ffprobeReport) {
if metadata == nil || report == nil {
return
}
if metadata.Title == "" {
if album := report.tag("album"); album != "" {
metadata.Title = album
}
}
// Subtitle joins the title Calibre-style, mirroring the ABS sidecar
// path — there is no separate subtitle column. Skip when the title
// already contains it (tools sometimes pre-join).
if subtitle := report.tag("----:com.apple.itunes:subtitle", "subtitle"); subtitle != "" && metadata.Title != "" &&
!strings.Contains(strings.ToLower(metadata.Title), strings.ToLower(subtitle)) {
metadata.Title = metadata.Title + ": " + subtitle
}
if metadata.Author == "" {
if artist := report.tag("album_artist"); artist != "" {
metadata.Author = artist
} else if artist := report.tag("artist"); artist != "" {
metadata.Author = artist
}
}
if len(metadata.Narrators) == 0 {
// Kept as ONE narrator string: composer values have no reliable
// multi-narrator delimiter, and splitting "Vance, Simon" would
// manufacture two fake people. Admins can edit the column.
if composer := report.tag("composer"); composer != "" {
metadata.Narrators = []string{composer}
if metadata.AudioMeta != nil && len(metadata.AudioMeta.Narrators) == 0 {
metadata.AudioMeta.Narrators = []string{composer}
}
}
}
if metadata.Series == "" {
if series := report.tag("grouping", "----:com.apple.itunes:series"); series != "" {
metadata.Series = series
if part := report.tag("----:com.apple.itunes:series-part"); part != "" {
if idx, err := strconv.ParseFloat(strings.TrimSpace(part), 32); err == nil {
metadata.SeriesNumber = int32(idx)
}
}
}
}
if metadata.ASIN == "" {
if asin := report.tag("----:com.apple.itunes:asin", "asin"); asin != "" {
metadata.ASIN = asin
if metadata.AudioMeta != nil && metadata.AudioMeta.ASIN == "" {
metadata.AudioMeta.ASIN = asin
}
}
}
if metadata.AudioMeta == nil || metadata.AudioMeta.Abridged == nil {
if !metadata.Abridged {
switch strings.ToLower(report.tag("----:com.apple.itunes:abridged")) {
case "1", "true", "yes":
metadata.Abridged = true
if metadata.AudioMeta != nil {
abridged := true
metadata.AudioMeta.Abridged = &abridged
}
}
}
}
if metadata.PublishDate.IsZero() {
if date := report.tag("date"); date != "" {
for _, layout := range []string{"2006-01-02", "2006-01", "2006"} {
if t, err := time.Parse(layout, strings.TrimSpace(date)); err == nil {
metadata.PublishDate = t
break
}
}
}
}
if metadata.Genre == "" {
metadata.Genre = report.tag("genre")
}
}
// audioDurationLabel renders the duration for logs/detail surfaces.
func audioDurationLabel(seconds float64) string {
if seconds <= 0 {
@@ -0,0 +1,173 @@
package services
// CUE sheet support: a `.cue` sidecar describing the chapter marks of a
// single big audio file (the classic CD-rip shape: one flac/mp3 + one
// cue). CUE is the LOWEST-priority chapter source — curated sidecar
// chapters and embedded m4b chapters outrank it — and a purely optional
// one: no cue, no chapters, no error.
import (
"bufio"
"os"
"path/filepath"
"strconv"
"strings"
)
// cueFramesPerSecond is the CD framing CUE INDEX timestamps use
// (MM:SS:FF with 75 frames per second).
const cueFramesPerSecond = 75.0
// cueTimeToSeconds converts a CUE INDEX timestamp ("03:25:12") to
// seconds. Minutes may exceed 59 and hours are folded into the minutes
// field per the spec (MM:SS:FF, MM unbounded).
func cueTimeToSeconds(v string) float64 {
parts := strings.Split(strings.TrimSpace(v), ":")
if len(parts) != 3 {
return 0
}
minutes, err1 := strconv.ParseFloat(strings.TrimSpace(parts[0]), 64)
seconds, err2 := strconv.ParseFloat(strings.TrimSpace(parts[1]), 64)
frames, err3 := strconv.ParseFloat(strings.TrimSpace(parts[2]), 64)
if err1 != nil || err2 != nil || err3 != nil {
return 0
}
return minutes*60 + seconds + frames/cueFramesPerSecond
}
// readCueChapters finds a CUE sheet describing the given audio file and
// returns its tracks as chapters (global seconds, ends derived from the
// next start; the last end stays 0 = unknown). A cue matches when its
// FILE directive names the audio file (basename, case-insensitive), or
// — for single-cue directories — when the directory holds exactly one.
func readCueChapters(path string) []AudioChapter {
dir := filepath.Dir(path)
base := strings.ToLower(filepath.Base(path))
cuePaths, err := filepath.Glob(filepath.Join(dir, "*.cue"))
if err != nil || len(cuePaths) == 0 {
return nil
}
var matched string
for _, cuePath := range cuePaths {
if referencedByCue(cuePath, base) {
matched = cuePath
break
}
}
if matched == "" && len(cuePaths) == 1 {
// Single-cue directory: the cue almost certainly describes the
// only audio file next to it.
matched = cuePaths[0]
}
if matched == "" {
return nil
}
return parseCueSheet(matched, base)
}
// referencedByCue reports whether the cue's FILE directive names the
// given audio basename.
func referencedByCue(cuePath, audioBase string) bool {
f, err := os.Open(cuePath)
if err != nil {
return false
}
defer func() { _ = f.Close() }()
scanner := bufio.NewScanner(f)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if len(line) >= 4 && strings.EqualFold(line[:4], "FILE") {
rest := strings.TrimSpace(line[4:])
rest = strings.Trim(rest, "\"")
// FILE "name" WAVE / FILE name MP3 — drop the type word.
if idx := strings.LastIndexAny(rest, " \t"); idx != -1 {
rest = strings.TrimSpace(rest[:idx])
}
rest = strings.Trim(rest, "\"")
return strings.EqualFold(filepath.Base(rest), audioBase)
}
}
return false
}
// parseCueSheet extracts TRACK/TITLE/INDEX 01 entries as chapters.
// INDEX timestamps are absolute from the file start per the spec; the
// pregap (INDEX 00) is ignored. Multi-file cues contribute only the
// TRACK blocks under the FILE directive naming our audio file (or all
// blocks when the cue declares no FILE at all).
func parseCueSheet(cuePath, audioBase string) []AudioChapter {
f, err := os.Open(cuePath)
if err != nil {
return nil
}
defer func() { _ = f.Close() }()
var chapters []AudioChapter
var current *AudioChapter
title := ""
inOurFile := true
flush := func() {
if current != nil && current.Start >= 0 {
current.Title = strings.TrimSpace(title)
if current.Title == "" {
current.Title = "Chapter " + strconv.Itoa(current.Index+1)
}
if n := len(chapters); n > 0 && current.Start >= chapters[n-1].Start {
chapters[n-1].End = current.Start
}
chapters = append(chapters, *current)
}
current = nil
title = ""
}
scanner := bufio.NewScanner(f)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
upper := strings.ToUpper(line)
switch {
case strings.HasPrefix(upper, "FILE"):
flush()
rest := strings.Trim(strings.TrimSpace(line[4:]), "\"")
if idx := strings.LastIndexAny(rest, " \t"); idx != -1 {
rest = strings.TrimSpace(rest[:idx])
}
rest = strings.Trim(rest, "\"")
inOurFile = strings.EqualFold(filepath.Base(rest), audioBase)
case strings.HasPrefix(upper, "TRACK"):
flush()
if !inOurFile {
continue
}
current = &AudioChapter{Index: len(chapters)}
if fields := strings.Fields(line); len(fields) >= 2 {
if n, err := strconv.Atoi(fields[1]); err == nil && n > 0 {
current.Index = n - 1
}
}
case strings.HasPrefix(upper, "TITLE") && current != nil:
if q1 := strings.Index(line, "\""); q1 != -1 {
if q2 := strings.LastIndex(line, "\""); q2 > q1 {
title = line[q1+1 : q2]
}
}
case strings.HasPrefix(upper, "INDEX"):
if current == nil || !inOurFile {
continue
}
if fields := strings.Fields(line); len(fields) >= 3 && fields[1] == "01" {
current.Start = cueTimeToSeconds(fields[2])
}
}
}
flush()
if len(chapters) == 0 {
return nil
}
return chapters
}
@@ -0,0 +1,277 @@
package services
// Unit tests for the exhaustive audiobook metadata ingestion: CUE
// sheets, m3u playlists, tag fallbacks (incl. Audible freeform atoms)
// and tag-based track ordering.
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestCueTimeToSeconds(t *testing.T) {
cases := map[string]float64{
"00:00:00": 0,
"01:00:00": 60,
"03:25:12": 3*60 + 25 + 12.0/75.0,
"75:00:37": 75*60 + 37.0/75.0, // minutes exceed 59 per spec
"not-a-time": 0,
"1:2": 0,
}
for in, want := range cases {
if got := cueTimeToSeconds(in); (got-want) > 0.001 || (want-got) > 0.001 {
t.Errorf("cueTimeToSeconds(%q) = %v, want %v", in, got, want)
}
}
}
func TestParseCueSheet(t *testing.T) {
dir := t.TempDir()
cuePath := filepath.Join(dir, "book.cue")
cue := `TITLE "Cue Book"
FILE "book.flac" WAVE
TRACK 01 AUDIO
TITLE "Opening"
INDEX 01 00:00:00
TRACK 02 AUDIO
TITLE "The Middle Bit"
INDEX 01 00:01:30
TRACK 03 AUDIO
TITLE "Closing"
INDEX 01 00:03:00
FILE "other.flac" WAVE
TRACK 01 AUDIO
TITLE "Foreign Track"
INDEX 01 00:00:00
`
if err := os.WriteFile(cuePath, []byte(cue), 0o644); err != nil {
t.Fatal(err)
}
chapters := parseCueSheet(cuePath, "book.flac")
if len(chapters) != 3 {
t.Fatalf("chapters = %d (%+v), want 3 (foreign FILE block excluded)", len(chapters), chapters)
}
if chapters[0].Title != "Opening" || chapters[0].Start != 0 {
t.Fatalf("first = %+v", chapters[0])
}
// MM:SS:FF: "00:01:30" = 1 second + 30/75 frames = 1.4s
if chapters[1].Start != 1.4 || chapters[1].End != 3.0 {
t.Fatalf("second = %+v", chapters[1])
}
if chapters[2].End != 0 {
t.Fatalf("last end must stay 0 (unknown): %+v", chapters[2])
}
}
func TestReadCueChaptersMatched(t *testing.T) {
dir := t.TempDir()
cue := "FILE \"audio.mp3\" WAVE\n TRACK 01 AUDIO\n TITLE \"One\"\n INDEX 01 00:00:00\n TRACK 02 AUDIO\n TITLE \"Two\"\n INDEX 01 00:00:10\n"
if err := os.WriteFile(filepath.Join(dir, "audio.cue"), []byte(cue), 0o644); err != nil {
t.Fatal(err)
}
chapters := readCueChapters(filepath.Join(dir, "audio.mp3"))
if len(chapters) != 2 {
t.Fatalf("chapters = %d", len(chapters))
}
// No cue at all → nil, never an error.
if chapters := readCueChapters(filepath.Join(dir, "other.mp3")); chapters != nil {
t.Fatalf("expected nil for missing cue, got %+v", chapters)
}
}
func TestFindAndApplyPlaylistHint(t *testing.T) {
dir := t.TempDir()
playlist := "#EXTM3U\n#EXTINF:120,Chapter Zero\n02_Second.mp3\n#EXTINF:60,Chapter One\n./01_First.mp3\nmissing.mp3\n"
if err := os.WriteFile(filepath.Join(dir, "book.m3u"), []byte(playlist), 0o644); err != nil {
t.Fatal(err)
}
// Exactly one playlist → hint; two → none.
hint := findPlaylistHint(dir)
if hint == nil {
t.Fatal("expected hint")
}
if err := os.WriteFile(filepath.Join(dir, "other.m3u8"), []byte("#EXTM3U\n"), 0o644); err != nil {
t.Fatal(err)
}
if findPlaylistHint(dir) != nil {
t.Fatal("multiple playlists must produce no hint")
}
abs := []string{
filepath.Join(dir, "01_First.mp3"),
filepath.Join(dir, "02_Second.mp3"),
filepath.Join(dir, "10_Later.mp3"), // not in playlist → appended last
}
ordered := hint.reorder(abs)
if len(ordered) != 3 || ordered[0] != abs[1] || ordered[1] != abs[0] || ordered[2] != abs[2] {
t.Fatalf("playlist order not applied: %v", ordered)
}
if got := hint.titleFor(abs[0]); got != "Chapter One" {
t.Fatalf("titleFor(./01_First.mp3) = %q", got)
}
if got := hint.titleFor(abs[1]); got != "Chapter Zero" {
t.Fatalf("titleFor(02_Second.mp3) = %q", got)
}
if got := hint.titleFor(abs[2]); got != "" {
t.Fatalf("unlisted track must have no title, got %q", got)
}
if h := (*playlistHint)(nil); h.titleFor(abs[0]) != "" {
t.Fatal("nil hint must be safe")
}
}
func syntheticReport(tags map[string]string) *ffprobeReport {
r := &ffprobeReport{}
r.Format.Tags = tags
return r
}
func TestApplyAudioTagFallbacks(t *testing.T) {
t.Run("full fill", func(t *testing.T) {
meta := &MediaMetadata{}
applyAudioTagFallbacks(meta, syntheticReport(map[string]string{
"album": "The Tagged Album",
"album_artist": "Tag Author",
"artist": "Wrong Author",
"composer": "Tag Narrator",
"grouping": "Tag Series",
"genre": "Audiobook",
"date": "2020",
"----:com.apple.iTunes:SUBTITLE": "Or, A Life",
"----:com.apple.iTunes:SERIES": "Freeform Series",
"----:com.apple.iTunes:SERIES-PART": "4",
"----:com.apple.iTunes:ASIN": "B0ASIN1234",
"----:com.apple.iTunes:ABRIDGED": "1",
}))
if meta.Title != "The Tagged Album: Or, A Life" {
t.Fatalf("title = %q", meta.Title)
}
if meta.Author != "Tag Author" {
t.Fatalf("author = %q", meta.Author)
}
if len(meta.Narrators) != 1 || meta.Narrators[0] != "Tag Narrator" {
t.Fatalf("narrators = %v", meta.Narrators)
}
// grouping outranks freeform SERIES for the series name only
// because it is checked first; both are tag-level fallbacks.
if meta.Series == "" {
t.Fatal("series not filled from tags")
}
if meta.SeriesNumber != 4 {
t.Fatalf("series number = %d", meta.SeriesNumber)
}
if meta.ASIN != "B0ASIN1234" {
t.Fatalf("asin = %q", meta.ASIN)
}
if !meta.Abridged {
t.Fatal("abridged not filled")
}
if meta.PublishDate.Year() != 2020 {
t.Fatalf("publish date = %v", meta.PublishDate)
}
if meta.Genre != "Audiobook" {
t.Fatalf("genre = %q", meta.Genre)
}
})
t.Run("fills only empty fields", func(t *testing.T) {
meta := &MediaMetadata{Title: "Sidecar Title", Author: "Sidecar Author", Narrators: []string{"Sidecar Narrator"}}
applyAudioTagFallbacks(meta, syntheticReport(map[string]string{
"album": "Tag Album", "artist": "Tag Artist", "composer": "Tag Narrator",
}))
if meta.Title != "Sidecar Title" || meta.Author != "Sidecar Author" {
t.Fatalf("tags must not override earlier sources: %+v", meta)
}
if len(meta.Narrators) != 1 || meta.Narrators[0] != "Sidecar Narrator" {
t.Fatalf("narrators = %v", meta.Narrators)
}
})
t.Run("explicit sidecar false survives abridged tag", func(t *testing.T) {
abridgedFalse := false
meta := &MediaMetadata{AudioMeta: &AudioMetadataPayload{Abridged: &abridgedFalse}}
applyAudioTagFallbacks(meta, syntheticReport(map[string]string{
"----:com.apple.iTunes:ABRIDGED": "1",
}))
if meta.Abridged {
t.Fatal("explicit sidecar false must not be overridden")
}
})
t.Run("nil report is a no-op", func(t *testing.T) {
meta := &MediaMetadata{}
applyAudioTagFallbacks(meta, nil)
if meta.Title != "" {
t.Fatal("nil report must change nothing")
}
})
}
func TestParseTagOrdinal(t *testing.T) {
cases := map[string]int{
"2": 2,
"3/9": 3,
" 12 ": 12,
"": 0,
"x": 0,
"0": 0,
"2/": 2,
"-1": 0,
}
for in, want := range cases {
if got := parseTagOrdinal(in); got != want {
t.Errorf("parseTagOrdinal(%q) = %d, want %d", in, got, want)
}
}
}
func TestOrderTracksByTags(t *testing.T) {
mk := func(disc, track int, rel string) audioTrackInfo {
return audioTrackInfo{disc: disc, trackNum: track, relPath: rel}
}
t.Run("tag order wins when complete", func(t *testing.T) {
probed := []audioTrackInfo{
mk(2, 1, "b/01.mp3"), mk(1, 2, "a/02.mp3"), mk(1, 10, "a/10.mp3"),
}
orderTracksByTags(probed)
want := []string{"a/02.mp3", "a/10.mp3", "b/01.mp3"}
for i, tr := range probed {
if tr.relPath != want[i] {
t.Fatalf("order = %v", probed)
}
}
})
t.Run("incomplete coverage keeps natural order", func(t *testing.T) {
probed := []audioTrackInfo{mk(1, 1, "a/01.mp3"), mk(0, 0, "a/02.mp3")}
orderTracksByTags(probed)
if probed[0].relPath != "a/01.mp3" || probed[1].relPath != "a/02.mp3" {
t.Fatalf("order = %v", probed)
}
})
}
func TestTagLookupCaseInsensitive(t *testing.T) {
r := syntheticReport(map[string]string{
"ALBUM": "Cased Album",
"----:com.apple.iTunes:ASIN": "B0CASE",
})
if got := r.tag("album"); got != "Cased Album" {
t.Fatalf("tag(album) = %q", got)
}
if got := r.tag("album", "title"); got != "Cased Album" {
t.Fatalf("multi-key lookup = %q", got)
}
if got := r.tag("----:COM.APPLE.ITUNES:asin"); !strings.EqualFold(got, "B0CASE") {
t.Fatalf("freeform case-insensitive lookup = %q", got)
}
if got := r.tag("missing"); got != "" {
t.Fatalf("missing key = %q", got)
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,203 @@
package services
// Unit tests for the multi-file audiobook grouping/manifest helpers
// (media_scanner_audio_multifile.go). Pure filesystem logic — no DB.
import (
"archive/zip"
"os"
"path/filepath"
"testing"
)
func TestNaturalLess(t *testing.T) {
cases := []struct {
a, b string
less bool
}{
{"01 Track.mp3", "02 Track.mp3", true},
{"02 Track.mp3", "10 Track.mp3", true}, // numeric, not lexicographic
{"10 Track.mp3", "9 Track.mp3", false},
{"Track 2", "Track 10", true},
{"abc", "abd", true},
{"01", "1", false}, // leading zeros compare equal → strict-less false
{"a1b", "a1c", true},
{"Disc 1", "Disc 2", true},
}
for _, tc := range cases {
if got := naturalLess(tc.a, tc.b); got != tc.less {
t.Errorf("naturalLess(%q, %q) = %v, want %v", tc.a, tc.b, got, tc.less)
}
}
}
func TestSortTracksNatural(t *testing.T) {
in := []string{"/x/10.mp3", "/x/2.mp3", "/x/1.mp3"}
sortTracksNatural(in)
want := []string{"/x/1.mp3", "/x/2.mp3", "/x/10.mp3"}
for i := range want {
if in[i] != want[i] {
t.Fatalf("sorted = %v, want %v", in, want)
}
}
}
// writeDummyTrack creates a file with plausible audio bytes (never probed
// in these tests — only structure matters).
func writeDummyTrack(t *testing.T, path string) {
t.Helper()
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte("ID3 fake audio bytes"), 0o644); err != nil {
t.Fatal(err)
}
}
func TestBookRootForTrackFile(t *testing.T) {
root := t.TempDir()
// Layout:
// root/Author/Title/01.mp3, 02.mp3 → one book at Title
// root/Author/Parts/CD1/01.mp3, CD2/01.mp3 → one book at Parts
// root/Author/Lone/book.mp3 → no folder book
writeDummyTrack(t, filepath.Join(root, "Author", "Title", "01.mp3"))
writeDummyTrack(t, filepath.Join(root, "Author", "Title", "02.mp3"))
writeDummyTrack(t, filepath.Join(root, "Author", "Parts", "CD1", "01.mp3"))
writeDummyTrack(t, filepath.Join(root, "Author", "Parts", "CD2", "01.mp3"))
writeDummyTrack(t, filepath.Join(root, "Author", "Lone", "book.mp3"))
t.Run("simple track folder", func(t *testing.T) {
got, tracks, ok := bookRootForTrackFile(filepath.Join(root, "Author", "Title", "02.mp3"), root)
if !ok {
t.Fatalf("expected folder book, got none")
}
if got != filepath.Join(root, "Author", "Title") {
t.Fatalf("root = %q", got)
}
if len(tracks) != 2 {
t.Fatalf("tracks = %v", tracks)
}
})
t.Run("disc folders merge into parent", func(t *testing.T) {
got, tracks, ok := bookRootForTrackFile(filepath.Join(root, "Author", "Parts", "CD1", "01.mp3"), root)
if !ok {
t.Fatalf("expected folder book")
}
if got != filepath.Join(root, "Author", "Parts") {
t.Fatalf("root = %q (disc folders must merge up)", got)
}
if len(tracks) != 2 {
t.Fatalf("tracks = %v", tracks)
}
})
t.Run("lone file is not a folder book", func(t *testing.T) {
_, _, ok := bookRootForTrackFile(filepath.Join(root, "Author", "Lone", "book.mp3"), root)
if ok {
t.Fatalf("single track must not form a folder book")
}
})
t.Run("sibling books never merge at author level", func(t *testing.T) {
writeDummyTrack(t, filepath.Join(root, "Author", "BookB", "01.mp3"))
writeDummyTrack(t, filepath.Join(root, "Author", "BookB", "02.mp3"))
got, _, ok := bookRootForTrackFile(filepath.Join(root, "Author", "Title", "01.mp3"), root)
if !ok || got != filepath.Join(root, "Author", "Title") {
t.Fatalf("root = %q, ok = %v; Title/BookB must stay separate", got, ok)
}
})
}
func TestBuildMultiFileAudioPayload(t *testing.T) {
tracks := []audioTrackInfo{
{relPath: "Author/Title/01.mp3", title: "Chapter One", duration: 100, size: 10},
{relPath: "Author/Title/02.mp3", title: "Chapter Two", duration: 50, size: 5},
}
payload := buildMultiFileAudioPayload("folder", nil, tracks)
if payload.Layout != "folder" {
t.Fatalf("layout = %q", payload.Layout)
}
if payload.DurationSeconds != 150 {
t.Fatalf("duration = %v", payload.DurationSeconds)
}
if len(payload.Tracks) != 2 {
t.Fatalf("tracks = %d", len(payload.Tracks))
}
if payload.Tracks[1].Start != 100 || payload.Tracks[1].End != 150 {
t.Fatalf("cumulative bounds wrong: %+v", payload.Tracks[1])
}
// No sidecar chapters: tracks ARE the chapters.
if len(payload.Chapters) != 2 || payload.Chapters[0].Start != 0 || payload.Chapters[1].Start != 100 {
t.Fatalf("chapters = %+v", payload.Chapters)
}
// Curated sidecar chapters keep priority; tracks still carry bounds.
sidecar := &AudioMetadataPayload{
Chapters: []AudioChapter{{Index: 0, Title: "Part I", Start: 0, End: 150}},
}
payload2 := buildMultiFileAudioPayload("zip", sidecar, tracks)
if len(payload2.Chapters) != 1 || payload2.Chapters[0].Title != "Part I" {
t.Fatalf("sidecar chapters must win: %+v", payload2.Chapters)
}
if len(payload2.Tracks) != 2 {
t.Fatalf("tracks lost: %+v", payload2.Tracks)
}
}
func TestZipAudioEntries(t *testing.T) {
zipPath := filepath.Join(t.TempDir(), "book.zip")
f, err := os.Create(zipPath)
if err != nil {
t.Fatal(err)
}
w := zip.NewWriter(f)
for _, name := range []string{
"__MACOSX/._junk.mp3",
".hidden.mp3",
"02 second.mp3",
"10 tenth.mp3",
"01 first.mp3",
"cover.jpg",
"notes.txt",
} {
if _, err := w.Create(name); err != nil {
t.Fatal(err)
}
}
if err := w.Close(); err != nil {
t.Fatal(err)
}
_ = f.Close()
entries, err := zipAudioEntries(zipPath)
if err != nil {
t.Fatal(err)
}
want := []string{"01 first.mp3", "02 second.mp3", "10 tenth.mp3"}
if len(entries) != len(want) {
t.Fatalf("entries = %v", entries)
}
for i, e := range entries {
if e.name != want[i] {
t.Fatalf("entry[%d] = %q, want %q", i, e.name, want[i])
}
}
}
func TestCleanTrackTitle(t *testing.T) {
cases := map[string]string{
"/x/01_Genesis001.mp3": "Genesis001",
"/x/01 - Track.mp3": "Track",
"/x/12_.Intro.mp3": "Intro",
"/x/Chapter 3.mp3": "Chapter 3",
"/x/JustTitle.m4b": "JustTitle",
"/x/01.mp3": "01", // number-only name: number stays the title
}
for in, want := range cases {
if got := cleanTrackTitle(in); got != want {
t.Errorf("cleanTrackTitle(%q) = %q, want %q", in, got, want)
}
}
}
+107 -25
View File
@@ -4,6 +4,7 @@ import (
"bookhoard/internal/database"
wsync "bookhoard/internal/sync"
"context"
"errors"
"fmt"
"net/http"
"path/filepath"
@@ -27,6 +28,13 @@ const (
JobStatusCancelled JobStatus = "cancelled"
)
// Sentinel errors surfaced by CancelJob so HTTP handlers can map them
// to 404/409 without string matching.
var (
ErrJobNotFound = errors.New("job not found")
ErrJobNotCancellable = errors.New("job cannot be cancelled")
)
type JobType string
const (
@@ -45,11 +53,29 @@ var WorkerInstance *Worker
func (w *Worker) Enqueue(job *Job) {
select {
case w.jobQueue <- job:
w.recordPending(job)
default:
fmt.Printf("Worker queue full, rejecting job: %s\n", job.ID)
}
}
// recordPending registers a queued job's pending result so the status
// endpoint and GetActiveScanJobs can report jobs still waiting for a
// worker, and so CancelJob can cancel them before execution starts.
// The entry is replaced when processJob dequeues the job.
func (w *Worker) recordPending(job *Job) {
libID, _ := job.Params["library_id"].(string)
w.mu.Lock()
defer w.mu.Unlock()
if _, exists := w.results[job.ID]; !exists {
w.results[job.ID] = &JobResult{
JobID: job.ID,
Status: JobStatusPending,
LibraryID: libID,
}
}
}
type Job struct {
ID string
Type JobType
@@ -146,8 +172,14 @@ type AnalyticsJobResult struct {
ReportPath string `json:"report_path"`
}
type Worker struct {
jobQueue chan *Job
results map[string]*JobResult
jobQueue chan *Job
results map[string]*JobResult
// cancels maps a running job's id to its per-job context cancel
// func, registered by processJob and removed at completion. Lets
// CancelJob stop a running job without disturbing siblings (all
// manual scans used to share the handler's process-lifetime
// context, so no targeted cancellation was possible).
cancels map[string]context.CancelFunc
connManager *wsync.ConnectionManager
mu sync.RWMutex
wg sync.WaitGroup
@@ -192,6 +224,7 @@ func NewWorkerWithConfig(numWorkers, queueCap int, connManager *wsync.Connection
w := &Worker{
jobQueue: make(chan *Job, queueCap),
results: make(map[string]*JobResult),
cancels: make(map[string]context.CancelFunc),
ctx: ctx,
cancel: cancel,
}
@@ -243,22 +276,40 @@ func (w *Worker) worker() {
func (w *Worker) processJob(job *Job) {
libID, _ := job.Params["library_id"].(string)
w.mu.Lock()
// A job cancelled while queued must never execute: honour the
// pending-cancel marker and leave its result untouched.
if result, exists := w.results[job.ID]; exists && result.Status == JobStatusCancelled {
w.mu.Unlock()
return
}
w.results[job.ID] = &JobResult{
JobID: job.ID,
Status: JobStatusRunning,
LibraryID: libID,
}
// Derive a per-job context so cancelling this job (CancelJob) stops
// only this job. The parent stays whatever the caller passed (the
// handler's context for manual scans, Background for auto-scans).
parent := job.Context
if parent == nil {
parent = context.Background()
}
jobCtx, cancel := context.WithCancel(parent)
job.Context = jobCtx
w.cancels[job.ID] = cancel
w.mu.Unlock()
defer func() {
w.mu.Lock()
delete(w.cancels, job.ID)
w.mu.Unlock()
cancel()
}()
started := time.Now()
job.StartedAt = &started
w.mu.Lock()
if result, exists := w.results[job.ID]; exists {
result.Status = JobStatusRunning
}
w.mu.Unlock()
var err error
var result interface{}
@@ -318,27 +369,43 @@ func (w *Worker) processJob(job *Job) {
}
}
// A cancelled job stopped mid-flight: keep its last real progress
// instead of claiming 100%, and drop the meaningless "context
// canceled" processing error.
finalProgress := 1.0
finalError := ""
if status == JobStatusCancelled {
finalError = ""
if old, exists := w.results[job.ID]; exists {
finalProgress = old.Progress
}
} else if err != nil {
finalError = err.Error()
}
w.results[job.ID] = &JobResult{
JobID: job.ID,
Status: status,
LibraryID: libID,
Error: func() string {
if err != nil {
return err.Error()
} else {
return ""
}
}(),
Result: result,
Progress: 1.0,
Error: finalError,
Result: result,
Progress: finalProgress,
FilesScanned: filesScanned,
NewItems: newItems,
Errors: errors,
}
if job.Type == JobTypeScan && w.connManager != nil && job.UserID != "" {
// Cancelled scans announce scan_cancelled so clients reset
// their scanning indicators; a "scan_complete" would read as
// a finished scan.
msgType := wsync.MessageTypeScanComplete
if status == JobStatusCancelled {
msgType = wsync.MessageTypeScanCancelled
}
w.connManager.BroadcastToUser(job.UserID, wsync.BroadcastMessage{
Type: wsync.MessageTypeScanComplete,
Type: msgType,
Data: map[string]interface{}{
"job_id": job.ID,
"files_scanned": filesScanned,
@@ -972,6 +1039,7 @@ func (w *Worker) EnqueueJob(job *Job) error {
select {
case w.jobQueue <- job:
w.recordPending(job)
return nil
case <-w.ctx.Done():
return fmt.Errorf("worker is shutting down")
@@ -1005,19 +1073,33 @@ func (w *Worker) GetActiveScanJobs() []JobResult {
return active
}
// CancelJob cancels a pending or running job by id. Pending jobs are
// marked cancelled so processJob skips them at dequeue (no context
// exists yet); running jobs have their per-job context cancelled — the
// job stops at its next context check and processJob finalises the
// status as cancelled.
func (w *Worker) CancelJob(jobID string) error {
w.mu.Lock()
defer w.mu.Unlock()
if result, exists := w.results[jobID]; exists {
if result.Status == JobStatusRunning || result.Status == JobStatusPending {
result.Status = JobStatusCancelled
return nil
}
return fmt.Errorf("job cannot be cancelled")
result, exists := w.results[jobID]
if !exists {
return ErrJobNotFound
}
return fmt.Errorf("job not found")
switch result.Status {
case JobStatusPending:
result.Status = JobStatusCancelled
return nil
case JobStatusRunning:
result.Status = JobStatusCancelled
if cancel, ok := w.cancels[jobID]; ok {
cancel()
}
return nil
default:
return ErrJobNotCancellable
}
}
func (w *Worker) Shutdown() {
+157
View File
@@ -0,0 +1,157 @@
package services
import (
"context"
"errors"
"testing"
)
// A job cancelled while still queued must never execute. processJob
// sees the pending-cancel marker at dequeue and leaves the result
// untouched; a scan without params would otherwise fail with
// "library_id required" and flip the status to failed — so an empty
// error field proves the job body never ran.
func TestCancelQueuedJobNeverExecutes(t *testing.T) {
// Zero workers: the job stays queued until processJob is driven by
// hand below.
w := NewWorkerWithConfig(0, 10, nil)
job := &Job{
ID: "queued-cancel",
Type: JobTypeScan,
Context: context.Background(),
Params: map[string]interface{}{},
}
if err := w.EnqueueJob(job); err != nil {
t.Fatalf("enqueue failed: %v", err)
}
// The pending result must exist before any worker picks the job up,
// so status polling can see (and cancel) queued jobs.
result, ok := w.GetJobStatus("queued-cancel")
if !ok {
t.Fatal("queued job has no result entry")
}
if result.Status != JobStatusPending {
t.Fatalf("expected pending status, got %q", result.Status)
}
if err := w.CancelJob("queued-cancel"); err != nil {
t.Fatalf("CancelJob failed: %v", err)
}
// Simulate the dequeue that races the cancellation.
w.processJob(job)
result, _ = w.GetJobStatus("queued-cancel")
if result.Status != JobStatusCancelled {
t.Fatalf("expected cancelled status after dequeue, got %q", result.Status)
}
if result.Error != "" {
t.Errorf("skipped job must carry no processing error, got %q", result.Error)
}
}
// Cancelling a running job must invoke its per-job context cancel func
// — the old CancelJob only flipped a map status and the job kept
// running to completion.
func TestCancelRunningJobCancelsContext(t *testing.T) {
w := NewWorker(1, nil)
ctx, cancel := context.WithCancel(context.Background())
job := &Job{ID: "running-cancel", Type: JobTypeScan, Context: ctx}
// Seed the state processJob establishes while a job runs.
w.mu.Lock()
w.results[job.ID] = &JobResult{JobID: job.ID, Status: JobStatusRunning}
w.cancels[job.ID] = cancel
w.mu.Unlock()
if err := w.CancelJob(job.ID); err != nil {
t.Fatalf("CancelJob failed: %v", err)
}
if ctx.Err() == nil {
t.Fatal("CancelJob must cancel the job's context")
}
result, _ := w.GetJobStatus(job.ID)
if result.Status != JobStatusCancelled {
t.Fatalf("expected cancelled status, got %q", result.Status)
}
}
func TestCancelJobErrors(t *testing.T) {
w := NewWorker(1, nil)
if err := w.CancelJob("missing-job"); !errors.Is(err, ErrJobNotFound) {
t.Errorf("unknown job: expected ErrJobNotFound, got %v", err)
}
w.mu.Lock()
w.results["done-job"] = &JobResult{JobID: "done-job", Status: JobStatusCompleted}
w.results["failed-job"] = &JobResult{JobID: "failed-job", Status: JobStatusFailed}
w.results["cancelled-job"] = &JobResult{JobID: "cancelled-job", Status: JobStatusCancelled}
w.mu.Unlock()
for _, id := range []string{"done-job", "failed-job", "cancelled-job"} {
if err := w.CancelJob(id); !errors.Is(err, ErrJobNotCancellable) {
t.Errorf("terminal job %q: expected ErrJobNotCancellable, got %v", id, err)
}
}
}
// Both enqueue paths must register a pending result: auto-scans arrive
// via Enqueue (fire-and-forget) and manual scans via EnqueueJob, and a
// queued job must be visible to status polling and cancellable before
// a worker dequeues it.
func TestEnqueueRecordsPendingResult(t *testing.T) {
w := NewWorkerWithConfig(0, 10, nil)
fireAndForget := &Job{
ID: "via-enqueue",
Type: JobTypeScan,
Context: context.Background(),
Params: map[string]interface{}{"library_id": "lib-auto"},
}
w.Enqueue(fireAndForget)
tracked := &Job{
ID: "via-enqueue-job",
Type: JobTypeScan,
Context: context.Background(),
Params: map[string]interface{}{"library_id": "lib-manual"},
}
if err := w.EnqueueJob(tracked); err != nil {
t.Fatalf("EnqueueJob failed: %v", err)
}
for id, libID := range map[string]string{
"via-enqueue": "lib-auto",
"via-enqueue-job": "lib-manual",
} {
result, ok := w.GetJobStatus(id)
if !ok {
t.Fatalf("%s: no pending result recorded", id)
}
if result.Status != JobStatusPending {
t.Errorf("%s: expected pending, got %q", id, result.Status)
}
if result.LibraryID != libID {
t.Errorf("%s: expected library_id %q, got %q", id, libID, result.LibraryID)
}
}
}
// A scan whose context is already cancelled must abort immediately
// with context.Canceled — before counting, walking, or running the
// archive pass. processJob classifies that error as a cancelled job.
func TestScanFoldersCancelledContext(t *testing.T) {
s := NewMediaScanner(nil)
ctx, cancel := context.WithCancel(context.Background())
cancel()
if err := s.ScanFolders(ctx); !errors.Is(err, context.Canceled) {
t.Fatalf("expected context.Canceled, got %v", err)
}
}
+1
View File
@@ -12,6 +12,7 @@ const (
FormatGroupReflowable FormatGroup = "reflowable"
FormatGroupFixedLayout FormatGroup = "fixed_layout"
FormatGroupComicArchive FormatGroup = "comic_archive"
FormatGroupAudiobook FormatGroup = "audiobook"
FormatGroupUnknown FormatGroup = "unknown"
)
+10 -1
View File
@@ -87,6 +87,15 @@ func ConvertProgress(sourceFormat, targetFormat FormatGroup, sourceData map[stri
result["panel"] = panel
}
case FormatGroupAudiobook:
// Audio progress is percentage + global seconds; there is no page
// grid to re-derive, so the percentage carries across as-is and
// the second-based locator follows when present.
result["percentage"] = percentage
if offset, ok := sourceData["audio_offset_seconds"].(int); ok {
result["audio_offset_seconds"] = offset
}
default:
return nil, fmt.Errorf("unsupported target format: %s", targetFormat)
}
@@ -108,7 +117,7 @@ func extractPercentage(sourceFormat FormatGroup, sourceData map[string]interface
}
}
case FormatGroupFixedLayout, FormatGroupComicArchive:
case FormatGroupFixedLayout, FormatGroupComicArchive, FormatGroupAudiobook:
if page, ok := sourceData["page"].(int); ok {
if total, ok := sourceData["total_pages"].(int); ok && total > 0 {
return float64(page) / float64(total)
+1
View File
@@ -20,6 +20,7 @@ const (
MessageTypeScanProgress = "scan_progress"
MessageTypeScanComplete = "scan_complete"
MessageTypeScanError = "scan_error"
MessageTypeScanCancelled = "scan_cancelled"
)
// BroadcastMessage represents a message to broadcast to connected clients
+8 -3
View File
@@ -97,9 +97,14 @@ templ Admin(user User, stats AdminStats) {
@Icon("refresh", "h-5 w-5")
Scanning Libraries
</h3>
<button @click="hideScanProgress()" class="icon-btn" aria-label="Close">
@Icon("close", "h-5 w-5")
</button>
<div class="flex items-center gap-2">
<button id="scan-cancel-all-btn" @click="cancelAllScans()" class="hidden icon-btn" aria-label="Cancel all scans" title="Cancel all scans">
@Icon("x-circle", "h-5 w-5")
</button>
<button @click="hideScanProgress()" class="icon-btn" aria-label="Close">
@Icon("close", "h-5 w-5")
</button>
</div>
</div>
<div class="mb-4">
<div class="flex justify-between text-sm mb-2">
+11 -3
View File
@@ -183,7 +183,15 @@ func Admin(user User, stats AdminStats) templ.Component {
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 17, "Scanning Libraries</h3><button @click=\"hideScanProgress()\" class=\"icon-btn\" aria-label=\"Close\">")
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 17, "Scanning Libraries</h3><div class=\"flex items-center gap-2\"><button id=\"scan-cancel-all-btn\" @click=\"cancelAllScans()\" class=\"hidden icon-btn\" aria-label=\"Cancel all scans\" title=\"Cancel all scans\">")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = Icon("x-circle", "h-5 w-5").Render(ctx, templ_7745c5c3_Buffer)
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 18, "</button> <button @click=\"hideScanProgress()\" class=\"icon-btn\" aria-label=\"Close\">")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
@@ -191,7 +199,7 @@ func Admin(user User, stats AdminStats) templ.Component {
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 18, "</button></div><div class=\"mb-4\"><div class=\"flex justify-between text-sm mb-2\"><span style=\"color: var(--text-secondary)\">Overall Progress</span> <span id=\"scan-progress-text\" style=\"color: var(--text-primary)\">0%</span></div><div class=\"w-full rounded-full h-3\" style=\"background-color: var(--surface-hover);\"><div id=\"scan-progress-bar\" class=\"h-3 rounded-full transition-all duration-500\" style=\"width: 0%; background-color: var(--accent);\"></div></div><div id=\"scan-status\" class=\"text-sm mt-2\" style=\"color: var(--text-secondary)\">Starting scan...</div></div><div id=\"library-progress-list\" class=\"space-y-3\"></div><div id=\"scan-results\" class=\"hidden mt-6 p-4 rounded-xl border\" style=\"background-color: var(--bg-primary); border-color: var(--border);\"><h4 class=\"font-semibold mb-2 flex items-center gap-2\" style=\"color: var(--status-success);\">")
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 19, "</button></div></div><div class=\"mb-4\"><div class=\"flex justify-between text-sm mb-2\"><span style=\"color: var(--text-secondary)\">Overall Progress</span> <span id=\"scan-progress-text\" style=\"color: var(--text-primary)\">0%</span></div><div class=\"w-full rounded-full h-3\" style=\"background-color: var(--surface-hover);\"><div id=\"scan-progress-bar\" class=\"h-3 rounded-full transition-all duration-500\" style=\"width: 0%; background-color: var(--accent);\"></div></div><div id=\"scan-status\" class=\"text-sm mt-2\" style=\"color: var(--text-secondary)\">Starting scan...</div></div><div id=\"library-progress-list\" class=\"space-y-3\"></div><div id=\"scan-results\" class=\"hidden mt-6 p-4 rounded-xl border\" style=\"background-color: var(--bg-primary); border-color: var(--border);\"><h4 class=\"font-semibold mb-2 flex items-center gap-2\" style=\"color: var(--status-success);\">")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
@@ -199,7 +207,7 @@ func Admin(user User, stats AdminStats) templ.Component {
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 19, "Scan Complete!</h4><div id=\"scan-results-content\" style=\"color: var(--text-secondary)\"></div><div class=\"mt-4 flex gap-2\"><button @click=\"window.location.reload()\" class=\"btn btn-primary\">Refresh to View Books</button> <button @click=\"hideScanProgress()\" class=\"btn btn-secondary\">Dismiss</button></div></div></div></div></main></body></html>")
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 20, "Scan Complete!</h4><div id=\"scan-results-content\" style=\"color: var(--text-secondary)\"></div><div class=\"mt-4 flex gap-2\"><button @click=\"window.location.reload()\" class=\"btn btn-primary\">Refresh to View Books</button> <button @click=\"hideScanProgress()\" class=\"btn btn-secondary\">Dismiss</button></div></div></div></div></main></body></html>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
+126 -6
View File
@@ -80,8 +80,14 @@ function showScanProgress(
<span class="font-medium" style="color: var(--text-primary)">
${libraryNames[jobId]}
</span>
<span id="status-${jobId}" class="text-sm" style="color: var(--text-secondary)">
Pending...
<span class="flex items-center gap-2">
<span id="status-${jobId}" class="text-sm" style="color: var(--text-secondary)">
Pending...
</span>
<button data-cancel-job="${jobId}"
class="text-xs px-2 py-0.5 rounded border shrink-0"
style="color: var(--text-secondary); border-color: var(--border);"
title="Cancel this scan">Cancel</button>
</span>
</div>
<div class="w-full bg-gray-700 rounded-full h-2">
@@ -95,6 +101,13 @@ function showScanProgress(
)
.join("");
// The rows are TS-rendered (sanctioned scan-progress island); bind the
// cancel buttons here since inline handlers can't reach module scope.
list.querySelectorAll<HTMLButtonElement>("[data-cancel-job]").forEach((btn) => {
btn.addEventListener("click", () => cancelScan(btn.dataset.cancelJob || ""));
});
setCancelAllVisible(true);
pollScanProgress(jobIds, libraryNames);
}
@@ -106,9 +119,12 @@ function pollScanProgress(
): void {
const token = localStorage.getItem("token");
const startTime = Date.now();
const cancelledJobs = new Set<string>();
scanPollInterval = setInterval(async () => {
let allComplete = true;
let totalProgress = 0;
let progressJobs = 0;
let totalFiles = 0;
let totalNewItems = 0;
let totalErrors = 0;
@@ -124,12 +140,25 @@ function pollScanProgress(
updateLibraryProgress(jobId, status);
totalProgress += status.progress || 0;
totalFiles += status.files_scanned || 0;
totalNewItems += status.new_items || 0;
totalErrors += status.errors || 0;
if (status.status !== "completed" && status.status !== "failed") {
// Cancelled jobs are terminal and keep their last real
// progress; average only jobs that can still advance so the
// bar can reach 100%.
if (status.status === "cancelled") {
cancelledJobs.add(jobId);
} else {
totalProgress += status.progress || 0;
progressJobs++;
}
if (
status.status !== "completed" &&
status.status !== "failed" &&
status.status !== "cancelled"
) {
allComplete = false;
}
}
@@ -139,7 +168,8 @@ function pollScanProgress(
}
// status.progress arrives 0-1 from the server; render as percent.
const overallProgress = Math.round((totalProgress / jobIds.length) * 100);
const overallProgress =
progressJobs > 0 ? Math.round((totalProgress / progressJobs) * 100) : 0;
const progressBar = document.getElementById(
"scan-progress-bar",
) as HTMLElement;
@@ -158,12 +188,14 @@ function pollScanProgress(
if (allComplete) {
clearInterval(scanPollInterval!);
setCancelAllVisible(false);
showScanResults(
jobIds.length,
totalFiles,
totalNewItems,
totalErrors,
elapsed,
cancelledJobs.size,
);
}
}, 2000);
@@ -187,9 +219,88 @@ function updateLibraryProgress(jobId: string, status: any): void {
running: `Scanning... ${percent}%`,
completed: `✓ Complete (${status.new_items || 0} items)`,
failed: `✗ Failed`,
cancelled: `✗ Cancelled`,
};
statusText.textContent = statusMessages[status.status] || status.status;
}
// Terminal jobs no longer need their per-row cancel button.
if (
status.status === "completed" ||
status.status === "failed" ||
status.status === "cancelled"
) {
document.querySelector(`[data-cancel-job="${jobId}"]`)?.remove();
}
}
// Cancel one scan job (POST /api/scanner/cancel/:jobId). The server
// aborts the walk at its next context check and skips the archive pass,
// so cancelling never archives unvisited files.
async function cancelScan(jobId: string): Promise<void> {
const token = localStorage.getItem("token");
if (!token) return;
const btn = document.querySelector<HTMLButtonElement>(
`[data-cancel-job="${jobId}"]`,
);
if (btn) {
btn.disabled = true;
btn.textContent = "Cancelling...";
}
try {
const resp = await fetch(`/api/scanner/cancel/${jobId}`, {
method: "POST",
headers: { Authorization: `Bearer ${token}` },
});
if (!resp.ok) {
const err = await resp.json().catch(() => ({}));
throw new Error(err.error || "Failed to cancel scan");
}
showToast("Scan cancellation requested", "success");
} catch (e) {
showToast(
e instanceof Error ? e.message : "Failed to cancel scan",
"error",
);
if (btn) {
btn.disabled = false;
btn.textContent = "Cancel";
}
}
}
// Cancel every active scan (admin): resolves the server's active job
// list so it also covers jobs this page never started (e.g. started
// before a refresh), then cancels each one.
async function cancelAllScans(): Promise<void> {
const token = localStorage.getItem("token");
if (!token) return;
try {
const resp = await fetch("/api/scanner/active", {
headers: { Authorization: `Bearer ${token}` },
});
if (!resp.ok) {
throw new Error("Failed to list active scans");
}
const data = await resp.json();
const jobs = (data.jobs || []) as Array<{ job_id: string }>;
if (jobs.length === 0) return;
await Promise.allSettled(jobs.map((job) => cancelScan(job.job_id)));
} catch (e) {
showToast(
e instanceof Error ? e.message : "Failed to cancel scans",
"error",
);
}
}
function setCancelAllVisible(visible: boolean): void {
const btn = document.getElementById("scan-cancel-all-btn");
if (btn) btn.classList.toggle("hidden", !visible);
}
// On page load: if a scan is already running (started before this page
@@ -247,6 +358,7 @@ function showScanResults(
items: number,
errors: number,
elapsed: number,
cancelled: number,
): void {
const resultsDiv = document.getElementById("scan-results") as HTMLElement;
const contentDiv = document.getElementById(
@@ -259,6 +371,7 @@ function showScanResults(
<p>• ${libCount} librar${libCount === 1 ? "y" : "ies"} scanned</p>
<p>• ${files} files processed</p>
<p>• ${items} new items added</p>
${cancelled > 0 ? `<p>• ${cancelled} scan${cancelled === 1 ? "" : "s"} cancelled</p>` : ""}
${errors > 0 ? `<p style="color: var(--accent);">• ${errors} errors</p>` : ""}
<p style="color: var(--text-secondary)">Completed in ${elapsed} seconds</p>
`;
@@ -266,7 +379,12 @@ function showScanResults(
resultsDiv.classList.remove("hidden");
const statusText = document.getElementById("scan-status") as HTMLElement;
if (statusText) statusText.textContent = "Scan complete!";
if (statusText) {
statusText.textContent =
cancelled > 0
? "Scan finished (some scans were cancelled)"
: "Scan complete!";
}
}
function hideScanProgress(): void {
@@ -274,6 +392,7 @@ function hideScanProgress(): void {
"scan-progress-container",
) as HTMLElement;
if (container) container.classList.add("hidden");
setCancelAllVisible(false);
}
async function loadWatchStatus(): Promise<void> {
@@ -523,6 +642,7 @@ Alpine.data("admin", () => ({
init() {
bindDeleteLibraryModalClose();
},
cancelAllScans,
checkActiveScans,
hideScanProgress,
loadWatchStatus,
+131 -11
View File
@@ -18,6 +18,36 @@ import { showToast } from "./toast";
type AudioChapter = { index: number; title: string; start: number; end: number };
// One playable file of an audiobook. Multi-file books (folder/zip
// layouts) carry several; single-file books carry exactly one spanning
// the whole file. start/end are GLOBAL seconds — the same coordinate
// system as chapters, bookmarks and audio_offset_seconds.
export type AudioTrack = { url: string; start: number; end: number; title?: string };
// trackIndexFor finds the track holding a global position: the last one
// whose start is at or before it (zero-duration/unknown-end tracks keep
// working because only start matters for membership).
export function trackIndexFor(tracks: AudioTrack[], globalSeconds: number): number {
for (let i = tracks.length - 1; i > 0; i -= 1) {
if (globalSeconds >= tracks[i].start) return i;
}
return 0;
}
// localOffsetFor converts a global position to seconds within track i.
export function localOffsetFor(tracks: AudioTrack[], index: number, globalSeconds: number): number {
const track = tracks[index];
if (!track) return Math.max(0, globalSeconds);
return Math.max(0, globalSeconds - track.start);
}
// globalSecondsFor converts (track i, local seconds) back to the global
// position — the inverse of localOffsetFor.
export function globalSecondsFor(tracks: AudioTrack[], index: number, localSeconds: number): number {
const track = tracks[index];
return (track ? track.start : 0) + Math.max(0, localSeconds);
}
const PROGRESS_INTERVAL_MS = 30_000;
interface AudioConfig {
@@ -57,6 +87,17 @@ Alpine.data("audioReader", () => ({
scrubbing: false,
scrubValue: 0,
// Track manifest: one AudioTrack per playable file. Single-file books
// (m4b/mp3) carry exactly one track whose URL is the whole file, so
// every code path below can treat playback as (track, local seconds)
// and map through globalSecondsFor/localOffsetFor.
fileUrl: "",
tracks: [] as AudioTrack[],
activeTrack: 0,
// Local seconds to apply to the <audio> element once the freshly
// switched track's metadata loads (src swaps reset currentTime).
pendingSeek: null as number | null,
// Bookmarks (A2d): time-positioned, with notes. Audio locator
// convention: position = "audio:<seconds>" (chapter_number filled for
// display); dedup + cross-device sync happen server-side.
@@ -105,17 +146,19 @@ Alpine.data("audioReader", () => ({
document.title = "REJ: " + (e.reason?.message || String(e.reason));
});
const body = document.body as HTMLElement;
const config = readConfig();
const mediaId = body.getAttribute("data-media-item-id") || "";
const title = body.getAttribute("data-title") || "";
const cover = body.getAttribute("data-cover") || "";
const fileUrl = body.getAttribute("data-file-url") || "";
this.mediaItemId = mediaId;
this.title = title;
this.fileUrl = fileUrl;
const audio = this.audio();
if (!audio) return;
audio.src = fileUrl;
// src is set by restorePosition once the track manifest is known —
// for multi-file books data-file-url (a directory or a zip root) is
// NOT directly playable. The catch branch below falls back to it.
audio.playbackRate = Number(this.speed) || 1;
audio.preload = "auto";
// Restored volume/mute from the settings strip's persisted state.
@@ -145,8 +188,17 @@ Alpine.data("audioReader", () => ({
}
// Element events.
audio.addEventListener("loadedmetadata", () => {
// A freshly switched track applies its queued position once the
// browser knows the duration; same-track seeks set currentTime
// directly in setActiveTrack.
if (this.pendingSeek != null) {
audio.currentTime = Math.max(0, this.pendingSeek);
this.pendingSeek = null;
}
});
audio.addEventListener("timeupdate", () => {
if (!this.scrubbing) this.currentTime = audio.currentTime;
if (!this.scrubbing) this.currentTime = this.globalPosition();
this.tickChapter();
// Honest session accounting: only ACTUAL PLAYING time counts. The
// prevSnapshot walk keeps internal seeks/crossfades from inflating
@@ -158,6 +210,13 @@ Alpine.data("audioReader", () => ({
}
this.prevTick = now;
});
audio.addEventListener("ended", () => {
// Natural end of track n: roll into n+1 and keep playing. At the
// last track the book is done — nothing to advance into.
if (this.activeTrack < this.tracks.length - 1) {
this.setActiveTrack(this.activeTrack + 1, 0, true);
}
});
audio.addEventListener("play", () => {
this.playing = true;
this.prevTick = performance.now();
@@ -199,6 +258,53 @@ Alpine.data("audioReader", () => ({
return document.getElementById("audio-element") as HTMLAudioElement | null;
},
// ---- track manifest (multi-file books) ----
buildTracks(data: Record<string, unknown>) {
const raw = (data.tracks || []) as Array<Record<string, unknown>>;
if (raw.length > 0) {
this.tracks = raw.map((t) => ({
url: String(t.url || ""),
start: Number(t.start) || 0,
end: Number(t.end) || 0,
title: t.title ? String(t.title) : undefined,
}));
} else {
// Single-file book: the whole file is the one track. end stays 0
// (unknown) — the browser's own duration governs the element and
// membership never needs it for a single track.
this.tracks = [{ url: this.fileUrl, start: 0, end: 0 }];
}
this.activeTrack = 0;
},
globalPosition(): number {
const audio = this.audio();
if (!audio) return this.currentTime;
return globalSecondsFor(this.tracks, this.activeTrack, audio.currentTime);
},
// setActiveTrack points the <audio> element at track i at localSeconds.
// Same-track moves set currentTime directly; a src swap queues the
// position for loadedmetadata (setting currentTime before the new
// metadata loads throws or silently resets).
setActiveTrack(index: number, localSeconds: number, resume: boolean) {
const audio = this.audio();
if (!audio || index < 0 || index >= this.tracks.length) return;
const track = this.tracks[index];
this.activeTrack = index;
if (audio.getAttribute("src") === track.url) {
this.pendingSeek = null;
audio.currentTime = Math.max(0, localSeconds);
} else {
this.pendingSeek = Math.max(0, localSeconds);
audio.setAttribute("src", track.url);
audio.playbackRate = Number(this.speed) || 1;
audio.load();
}
if (resume) audio.play().catch(() => showToast("Playback failed to start", "error"));
},
async restorePosition() {
try {
const [chaptersResp, progress] = await Promise.all([
@@ -212,6 +318,11 @@ Alpine.data("audioReader", () => ({
const data = await chaptersResp.json();
this.chapters = (data.chapters || []) as AudioChapter[];
this.bookDuration = data.duration_seconds || this.chapterEndEstimate();
this.buildTracks(data);
const audio = this.audio();
if (audio && !audio.getAttribute("src") && this.tracks[0]) {
audio.setAttribute("src", this.tracks[0].url);
}
}
if (progress.ok) {
const p = await progress.json();
@@ -233,7 +344,13 @@ Alpine.data("audioReader", () => ({
this.tickChapter();
}
} catch {
// Offline start: still playable; sync resumes when reachable.
// Offline start: still playable from the plain file URL (single-
// file books only; multi-file books need the manifest).
const audio = this.audio();
if (audio && !audio.getAttribute("src") && this.fileUrl) {
this.tracks = [{ url: this.fileUrl, start: 0, end: 0 }];
audio.setAttribute("src", this.fileUrl);
}
}
},
@@ -282,14 +399,16 @@ Alpine.data("audioReader", () => ({
const audio = this.audio();
if (!audio) return;
const target = Number(seconds);
if (Number.isFinite(target)) audio.currentTime = Math.max(0, target);
this.currentTime = audio.currentTime;
if (!Number.isFinite(target)) return;
// Global seconds in: resolve the owning track and move within it.
const index = trackIndexFor(this.tracks, target);
this.setActiveTrack(index, localOffsetFor(this.tracks, index, target), false);
this.currentTime = target;
},
skip(direction: 1 | -1) {
const audio = this.audio();
if (!audio) return;
audio.currentTime = Math.max(0, audio.currentTime + direction * (Number(this.skipSeconds) || 10));
const target = Math.max(0, this.globalPosition() + direction * (Number(this.skipSeconds) || 10));
this.seekTo(target);
},
stepChapter(delta: 1 | -1) {
@@ -406,7 +525,7 @@ Alpine.data("audioReader", () => ({
async saveBookmark() {
const audio = this.audio();
if (!audio) return;
const at = Math.round(audio.currentTime);
const at = Math.round(this.globalPosition());
const chapter = this.chapters[this.currentChapterIdx];
const title = chapter
? `${chapter.title || `Chapter ${chapter.index + 1}`} — ${fmtTime(at)}`
@@ -502,7 +621,8 @@ Alpine.data("audioReader", () => ({
if (this.chaptersDuration() <= 0) return;
const audio = this.audio();
if (!audio) return;
const at = audio.currentTime;
// Global book position, whatever track is physically playing.
const at = this.globalPosition();
let chapter = 0;
let offset = 0;
for (let i = 0; i < this.chapters.length; i += 1) {
+5
View File
@@ -92,6 +92,11 @@ const initializeScanListener = function (this: any) {
this.scanning = false;
this.scanProgress = 0;
break;
case "scan_cancelled":
// A cancelled scan is not an error: stop the spinner quietly.
this.scanning = false;
this.scanProgress = 0;
break;
}
};
+6 -1
View File
@@ -432,10 +432,15 @@ function initSetup(): void {
running: `Scanning... ${Math.round((status.progress || 0) * 100)}%`,
completed: `✓ Complete (${status.new_items || 0} items)`,
failed: "✗ Failed",
cancelled: "✗ Cancelled",
};
job.statusText = statusMessages[status.status] || status.status;
if (status.status !== "completed" && status.status !== "failed") {
if (
status.status !== "completed" &&
status.status !== "failed" &&
status.status !== "cancelled"
) {
allComplete = false;
}
} else {