feat(audiobooks): multi-file books (folder-of-tracks + zip) + exhaustive metadata ingestion
Multi-file book ingest: a directory of ≥2 audio files (recursively —
disc-part folders like CD1/Disc 2 merge upward, siblings never merge)
ingests as ONE book whose file_path is the book-root directory; a zip
of audio tracks is its own book (file_path = the archive). Both write
the documented chapters abstraction {index,title,start,end} into
audio_metadata and add additive keys: layout ("folder"|"zip") and a
tracks[] manifest (per-track path, global-seconds start/end, duration,
size). Pre-grouping one-track strays under a book root are absorbed on
rescan (annotations cascade). Rescans take a stat-only fast path
(track count + total size + known durations unchanged → no re-probe —
a 1189-track book must not re-ffprobe every auto-scan); a zero-duration
book counts as changed so a ffprobe-equipped rescan fills durations in.
Folder-book roots count as present in the archive lifecycle pass, and
per-item rescan (RescanMediaItem) routes multi-file books through a
book-level refresh — the generic updater would read the root/zip as if
it were the book file and clobber format_group with "unknown".
Serving: GET /api/media-items/:id/chapters gains additive layout +
tracks keys (urls → the new endpoint). New GET
/api/media-items/:id/audio/:trackIndex streams one track — folder
tracks served from the real file (full Range via ServeContent), zip
tracks via a reopen-and-skip ReadSeeker over the entry (deflate has no
random access; acceptable at audio bitrates; stored entries stay
section readers). Both endpoints gate on library visibility (403 on
hidden libraries — family-lens rule). ServeFile and OPDS DownloadBook
refuse directory paths (folder books have no single-file download).
Web player (next commit) consumes the manifest; progress/bookmarks/
deep-links keep their global-seconds coordinates, so the server-side
progress model is untouched. sync learns FormatGroupAudiobook
(percentage passthrough in ConvertProgress); detectFormatType stops
reporting "unknown" for audio.
Metadata ingestion (Batch B) — read everything available, degrade
gracefully, documented precedence sidecars → folder structure →
embedded tags → filename: ffprobe tags are now a case-insensitive
lookup map (freeform keys pass through verbatim); composer fills
narrators (Audible's own convention, kept as ONE string — no reliable
multi-narrator delimiter); grouping + Audible freeform atoms
(SERIES/SERIES-PART/SUBTITLE/ASIN/ABRIDGED) fill series/index/
subtitle-join/ASIN/abridged (explicit sidecar abridged-false is
guarded); date/genre fall back to tags; album/album_artist/artist
fill title/author. Genre persists at create for all formats.
Embedded cover art (mp3 APIC, m4b covr, flac picture) extracts via one
optional ffmpeg call as a "<book>.cover.jpg" sibling — sidecar covers
win. CUE sheets (FILE-matched, INDEX 01 at 75fps) are the lowest-
priority chapter source under sidecars and embedded m4b chapters. A
lone m3u/m3u8 in a book root imposes playlist ordering + #EXTINF
names; (disc, track) tags reorder tracks when coverage is complete.
extractMetadata's default branch no longer pre-fills the filename as
title (tags must outrank it) — final filename fallbacks now exist on
BOTH the create and rescan paths (TestRescanKeepsFilenameTitle pins
that rescans cannot blank a title); tagless zip books take the zip's
own name, not their parent folder's.
Accepted containers widen to .m4b .m4a .mp3 .flac .zip .opus .ogg .oga
.aac .wav with real media types registered (audio/ogg per RFC 7845 for
Opus-in-Ogg) — all browser-playable; .wma deliberately excluded.
Tests: unit tests for CUE parsing, playlist hints, tag fallbacks
(synthetic ffprobe reports — ffmpeg cannot write Audible freeform
atoms), ordinal parsing, tag ordering, natural sort, grouping rules;
integration suite (11 tests) over the live dev DB: folder/zip ingest +
idempotency, stray absorption, chapters manifest, per-track streaming
with Range, hidden-library 403s, embedded covers + sidecar-wins, CUE
chapters, embedded m4b chapters, playlist ordering, tag fallbacks, all
five new containers with content-type assertions, rescan title pin.
Fixture media is real committed-format binary content living untracked
under internal/services/testdata/ (gitignored by policy; recipes in
docs/contributing/development.md → Media fixtures). Test image now
ships ffmpeg to match the runtime's audio toolchain.
This commit is contained in:
1 parent
24228e7147
commit
6216bd9bcf
26 files changed
+3419
-60
No files matched your search
+231
-4
@@ -1,6 +1,7 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/services"
|
||||
wsync "bookhoard/internal/sync"
|
||||
@@ -2550,9 +2551,16 @@ func (mh *MediaHandler) ServeFile(c *echo.Context) error {
|
||||
}
|
||||
|
||||
// Check if file exists
|
||||
if _, err := os.Stat(fullPath); os.IsNotExist(err) {
|
||||
fullInfo, err := os.Stat(fullPath)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "file not found"})
|
||||
}
|
||||
// Folder-of-tracks audiobooks store their book root DIRECTORY in
|
||||
// file_path; there is no single file to serve and ServeFile must not
|
||||
// fall through to directory listing behavior.
|
||||
if fullInfo.IsDir() {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "multi-file books have no single download"})
|
||||
}
|
||||
|
||||
ext := strings.ToLower(filepath.Ext(fullPath))
|
||||
contentType := services.MimeTypes[ext]
|
||||
@@ -2648,9 +2656,12 @@ func (mh *MediaHandler) saveCoverImage(c echo.Context, mediaUUID uuid.UUID, file
|
||||
|
||||
// GetAudioChapters handles GET /api/media-items/:id/chapters — the
|
||||
// format-abstraction chapter list for audiobooks (media_items
|
||||
// .audio_metadata). Embedded m4b chapters and folder-as-book files write
|
||||
// the same shape. Empty list when the item isn't audio or hasn't been
|
||||
// probed.
|
||||
// .audio_metadata). Embedded m4b chapters, folder-of-tracks books and zip
|
||||
// books write the same shape. Multi-file books additionally carry
|
||||
// `layout` and a `tracks` manifest whose URLs point at the per-track
|
||||
// streaming endpoint; single-file books keep the empty layout their
|
||||
// existing consumers expect. Empty list when the item isn't audio or
|
||||
// hasn't been probed.
|
||||
func (mh *MediaHandler) GetAudioChapters(c *echo.Context) error {
|
||||
mediaID := c.Param("id")
|
||||
mediaUUID, err := uuid.Parse(mediaID)
|
||||
@@ -2662,6 +2673,20 @@ func (mh *MediaHandler) GetAudioChapters(c *echo.Context) error {
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "book not found"})
|
||||
}
|
||||
if !item.LibraryID.Valid {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "book has no library"})
|
||||
}
|
||||
|
||||
// Library visibility gate (family-lens rule): chapter lists are media
|
||||
// metadata — a hidden library must not leak them any more than its
|
||||
// file bytes.
|
||||
visibleSet, _, err := mh.visibleLibrarySet(c)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to check library access"})
|
||||
}
|
||||
if !visibleSet[uuid.UUID(item.LibraryID.Bytes).String()] {
|
||||
return c.JSON(http.StatusForbidden, map[string]string{"error": "library not accessible"})
|
||||
}
|
||||
|
||||
var payload services.AudioMetadataPayload
|
||||
if len(item.AudioMetadata) > 0 {
|
||||
@@ -2670,10 +2695,212 @@ func (mh *MediaHandler) GetAudioChapters(c *echo.Context) error {
|
||||
}
|
||||
}
|
||||
|
||||
tracks := make([]map[string]interface{}, 0, len(payload.Tracks))
|
||||
for _, tr := range payload.Tracks {
|
||||
tracks = append(tracks, map[string]interface{}{
|
||||
"index": tr.Index,
|
||||
"url": fmt.Sprintf("/api/media-items/%s/audio/%d", mediaID, tr.Index),
|
||||
"title": tr.Title,
|
||||
"path": tr.Path,
|
||||
"start": tr.Start,
|
||||
"end": tr.End,
|
||||
"duration_seconds": tr.DurationSeconds,
|
||||
"size_bytes": tr.SizeBytes,
|
||||
})
|
||||
}
|
||||
|
||||
return c.JSON(http.StatusOK, map[string]interface{}{
|
||||
"media_item_id": item.ID,
|
||||
"chapters": payload.Chapters,
|
||||
"duration_seconds": payload.DurationSeconds,
|
||||
"codec_profile": payload.CodecProfile,
|
||||
"layout": payload.Layout,
|
||||
"tracks": tracks,
|
||||
})
|
||||
}
|
||||
|
||||
// GetAudioTrack handles GET /api/media-items/:id/audio/:trackIndex — the
|
||||
// per-track byte stream for multi-file audiobooks (folder books serve the
|
||||
// real file; zip books extract the entry in place). Library visibility is
|
||||
// the permission gate, same as ServeFile.
|
||||
func (mh *MediaHandler) GetAudioTrack(c *echo.Context) error {
|
||||
mediaUUID, err := uuid.Parse(c.Param("id"))
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid media item id"})
|
||||
}
|
||||
trackIndex, err := strconv.Atoi(c.Param("trackIndex"))
|
||||
if err != nil || trackIndex < 0 {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid track index"})
|
||||
}
|
||||
|
||||
item, err := mh.db.GetMediaItem(c.Request().Context(), pgtype.UUID{Bytes: mediaUUID, Valid: true})
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "book not found"})
|
||||
}
|
||||
if !item.LibraryID.Valid {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "book has no library"})
|
||||
}
|
||||
|
||||
// Library visibility gate (family-lens rule): a hidden library means
|
||||
// its audiobooks' bytes are not reachable either.
|
||||
visibleSet, _, err := mh.visibleLibrarySet(c)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to check library access"})
|
||||
}
|
||||
if !visibleSet[uuid.UUID(item.LibraryID.Bytes).String()] {
|
||||
return c.JSON(http.StatusForbidden, map[string]string{"error": "library not accessible"})
|
||||
}
|
||||
|
||||
var payload services.AudioMetadataPayload
|
||||
if len(item.AudioMetadata) > 0 {
|
||||
if err := json.Unmarshal(item.AudioMetadata, &payload); err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to parse audio metadata"})
|
||||
}
|
||||
}
|
||||
if trackIndex >= len(payload.Tracks) {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "track not found"})
|
||||
}
|
||||
track := payload.Tracks[trackIndex]
|
||||
|
||||
switch payload.Layout {
|
||||
case "folder":
|
||||
fullPath, err := mh.getFullFilePath(c.Request().Context(), item.LibraryID, track.Path)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "track file not found"})
|
||||
}
|
||||
f, err := os.Open(fullPath)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "track file not found"})
|
||||
}
|
||||
defer func() { _ = f.Close() }()
|
||||
info, err := f.Stat()
|
||||
if err != nil || info.IsDir() {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "track file not found"})
|
||||
}
|
||||
c.Response().Header().Set("Content-Type", audioTrackContentType(track.Path))
|
||||
c.Response().Header().Set("Cache-Control", "private, max-age=86400")
|
||||
// http.ServeContent wants a ReadSeeker for Range support; the os
|
||||
// file provides both.
|
||||
http.ServeContent(c.Response(), c.Request(), filepath.Base(fullPath), info.ModTime(), f)
|
||||
return nil
|
||||
|
||||
case "zip":
|
||||
zipPath, err := mh.getFullFilePath(c.Request().Context(), item.LibraryID, item.FilePath)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "archive not found"})
|
||||
}
|
||||
seeker, err := newZipEntrySeeker(zipPath, track.Path)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "track not found in archive"})
|
||||
}
|
||||
defer func() { _ = seeker.Close() }()
|
||||
c.Response().Header().Set("Content-Type", audioTrackContentType(track.Path))
|
||||
c.Response().Header().Set("Cache-Control", "private, max-age=86400")
|
||||
// Zip entries re-open+skip for backward seeks (deflate has no
|
||||
// random access), which audio bitrates make cheap enough.
|
||||
http.ServeContent(c.Response(), c.Request(), filepath.Base(track.Path), time.Time{}, seeker)
|
||||
return nil
|
||||
|
||||
default:
|
||||
// Single-file books stream through the /uploads path; the track
|
||||
// endpoint has nothing to slice.
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "book has no track manifest"})
|
||||
}
|
||||
}
|
||||
|
||||
// audioTrackContentType maps a track path/entry name to its mime type.
|
||||
func audioTrackContentType(name string) string {
|
||||
if m := services.MimeTypes[strings.ToLower(filepath.Ext(name))]; m != "" {
|
||||
return m
|
||||
}
|
||||
return "audio/mpeg"
|
||||
}
|
||||
|
||||
// zipEntrySeeker is an io.ReadSeeker over one zip entry. Forward seeks
|
||||
// discard bytes in the open stream; backward seeks re-open the entry and
|
||||
// skip — the only option over a deflate stream, and cheap at audio
|
||||
// bitrates. Owns the archive reader for the life of the request.
|
||||
type zipEntrySeeker struct {
|
||||
zrc *zip.ReadCloser
|
||||
rc io.ReadCloser
|
||||
pos int64
|
||||
size int64
|
||||
entryFn func() (io.ReadCloser, error)
|
||||
}
|
||||
|
||||
func newZipEntrySeeker(zipPath, entryName string) (*zipEntrySeeker, error) {
|
||||
zrc, err := zip.OpenReader(zipPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var entry *zip.File
|
||||
for _, f := range zrc.File {
|
||||
if filepath.ToSlash(f.Name) == filepath.ToSlash(entryName) {
|
||||
entry = f
|
||||
break
|
||||
}
|
||||
}
|
||||
if entry == nil {
|
||||
_ = zrc.Close()
|
||||
return nil, fmt.Errorf("entry %q not found", entryName)
|
||||
}
|
||||
open := func() (io.ReadCloser, error) { return entry.Open() }
|
||||
rc, err := open()
|
||||
if err != nil {
|
||||
_ = zrc.Close()
|
||||
return nil, err
|
||||
}
|
||||
return &zipEntrySeeker{
|
||||
zrc: zrc,
|
||||
rc: rc,
|
||||
size: int64(entry.UncompressedSize64),
|
||||
entryFn: open,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (z *zipEntrySeeker) Read(p []byte) (int, error) {
|
||||
n, err := z.rc.Read(p)
|
||||
z.pos += int64(n)
|
||||
return n, err
|
||||
}
|
||||
|
||||
func (z *zipEntrySeeker) Seek(offset int64, whence int) (int64, error) {
|
||||
var target int64
|
||||
switch whence {
|
||||
case io.SeekStart:
|
||||
target = offset
|
||||
case io.SeekCurrent:
|
||||
target = z.pos + offset
|
||||
case io.SeekEnd:
|
||||
target = z.size + offset
|
||||
default:
|
||||
return z.pos, fmt.Errorf("invalid whence %d", whence)
|
||||
}
|
||||
if target < 0 {
|
||||
return z.pos, fmt.Errorf("negative seek target %d", target)
|
||||
}
|
||||
if target < z.pos {
|
||||
// Deflate streams can't rewind: restart the entry and skip.
|
||||
if err := z.rc.Close(); err != nil {
|
||||
return z.pos, err
|
||||
}
|
||||
rc, err := z.entryFn()
|
||||
if err != nil {
|
||||
return z.pos, err
|
||||
}
|
||||
z.rc = rc
|
||||
z.pos = 0
|
||||
}
|
||||
if remaining := target - z.pos; remaining > 0 {
|
||||
if _, err := io.CopyN(io.Discard, z.rc, remaining); err != nil {
|
||||
return z.pos, err
|
||||
}
|
||||
z.pos = target
|
||||
}
|
||||
return z.pos, nil
|
||||
}
|
||||
|
||||
func (z *zipEntrySeeker) Close() error {
|
||||
_ = z.rc.Close()
|
||||
return z.zrc.Close()
|
||||
}
|
||||
@@ -916,9 +916,16 @@ func (h *OPDSHandler) DownloadBook(c *echo.Context) error {
|
||||
}
|
||||
|
||||
// Check if file exists
|
||||
if _, err := os.Stat(filePath); os.IsNotExist(err) {
|
||||
fileInfo, err := os.Stat(filePath)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "file not found"})
|
||||
}
|
||||
// Folder-of-tracks audiobooks point at a book root DIRECTORY: there
|
||||
// is no single file to stream for OPDS clients (a directory fd reads
|
||||
// EISDIR). Zip books are real files and keep working.
|
||||
if fileInfo.IsDir() {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "multi-file books have no single download"})
|
||||
}
|
||||
|
||||
// Open file
|
||||
file, err := os.Open(filePath)
|
||||
@@ -927,11 +934,7 @@ func (h *OPDSHandler) DownloadBook(c *echo.Context) error {
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
// Get file info
|
||||
fileInfo, err := file.Stat()
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to get file info"})
|
||||
}
|
||||
// Get file info (fileInfo already statted above)
|
||||
|
||||
// Set headers
|
||||
filename := filepath.Base(filePath)
|
||||
|
||||
Reference in new issue
Block a user