John O'Keefe
6216bd9bcf
Multi-file book ingest: a directory of ≥2 audio files (recursively —
disc-part folders like CD1/Disc 2 merge upward, siblings never merge)
ingests as ONE book whose file_path is the book-root directory; a zip
of audio tracks is its own book (file_path = the archive). Both write
the documented chapters abstraction {index,title,start,end} into
audio_metadata and add additive keys: layout ("folder"|"zip") and a
tracks[] manifest (per-track path, global-seconds start/end, duration,
size). Pre-grouping one-track strays under a book root are absorbed on
rescan (annotations cascade). Rescans take a stat-only fast path
(track count + total size + known durations unchanged → no re-probe —
a 1189-track book must not re-ffprobe every auto-scan); a zero-duration
book counts as changed so a ffprobe-equipped rescan fills durations in.
Folder-book roots count as present in the archive lifecycle pass, and
per-item rescan (RescanMediaItem) routes multi-file books through a
book-level refresh — the generic updater would read the root/zip as if
it were the book file and clobber format_group with "unknown".
Serving: GET /api/media-items/:id/chapters gains additive layout +
tracks keys (urls → the new endpoint). New GET
/api/media-items/:id/audio/:trackIndex streams one track — folder
tracks served from the real file (full Range via ServeContent), zip
tracks via a reopen-and-skip ReadSeeker over the entry (deflate has no
random access; acceptable at audio bitrates; stored entries stay
section readers). Both endpoints gate on library visibility (403 on
hidden libraries — family-lens rule). ServeFile and OPDS DownloadBook
refuse directory paths (folder books have no single-file download).
Web player (next commit) consumes the manifest; progress/bookmarks/
deep-links keep their global-seconds coordinates, so the server-side
progress model is untouched. sync learns FormatGroupAudiobook
(percentage passthrough in ConvertProgress); detectFormatType stops
reporting "unknown" for audio.
Metadata ingestion (Batch B) — read everything available, degrade
gracefully, documented precedence sidecars → folder structure →
embedded tags → filename: ffprobe tags are now a case-insensitive
lookup map (freeform keys pass through verbatim); composer fills
narrators (Audible's own convention, kept as ONE string — no reliable
multi-narrator delimiter); grouping + Audible freeform atoms
(SERIES/SERIES-PART/SUBTITLE/ASIN/ABRIDGED) fill series/index/
subtitle-join/ASIN/abridged (explicit sidecar abridged-false is
guarded); date/genre fall back to tags; album/album_artist/artist
fill title/author. Genre persists at create for all formats.
Embedded cover art (mp3 APIC, m4b covr, flac picture) extracts via one
optional ffmpeg call as a "<book>.cover.jpg" sibling — sidecar covers
win. CUE sheets (FILE-matched, INDEX 01 at 75fps) are the lowest-
priority chapter source under sidecars and embedded m4b chapters. A
lone m3u/m3u8 in a book root imposes playlist ordering + #EXTINF
names; (disc, track) tags reorder tracks when coverage is complete.
extractMetadata's default branch no longer pre-fills the filename as
title (tags must outrank it) — final filename fallbacks now exist on
BOTH the create and rescan paths (TestRescanKeepsFilenameTitle pins
that rescans cannot blank a title); tagless zip books take the zip's
own name, not their parent folder's.
Accepted containers widen to .m4b .m4a .mp3 .flac .zip .opus .ogg .oga
.aac .wav with real media types registered (audio/ogg per RFC 7845 for
Opus-in-Ogg) — all browser-playable; .wma deliberately excluded.
Tests: unit tests for CUE parsing, playlist hints, tag fallbacks
(synthetic ffprobe reports — ffmpeg cannot write Audible freeform
atoms), ordinal parsing, tag ordering, natural sort, grouping rules;
integration suite (11 tests) over the live dev DB: folder/zip ingest +
idempotency, stray absorption, chapters manifest, per-track streaming
with Range, hidden-library 403s, embedded covers + sidecar-wins, CUE
chapters, embedded m4b chapters, playlist ordering, tag fallbacks, all
five new containers with content-type assertions, rescan title pin.
Fixture media is real committed-format binary content living untracked
under internal/services/testdata/ (gitignored by policy; recipes in
docs/contributing/development.md → Media fixtures). Test image now
ships ffmpeg to match the runtime's audio toolchain.
84 lines
4.5 KiB
Markdown
84 lines
4.5 KiB
Markdown
# Audiobook metadata: sources & precedence
|
|
|
|
Every audiobook ingestion path (single-file, folder-of-tracks, zip-of-
|
|
audio) fills the same fields from the same ranked sources. Each source
|
|
**only fills fields the earlier ones left empty** — curated data always
|
|
outranks guesses — and every source is optional: a book still ingests
|
|
(title from the filename) when all of them are absent. Admin edits made
|
|
after ingest are protected by the metadata-override mechanism on
|
|
rescans, regardless of source quality.
|
|
|
|
## Book-level metadata (title, author, narrators, series, …)
|
|
|
|
| Rank | Source | Feeds |
|
|
|---|---|---|
|
|
| 1 | **Sidecars** — `metadata.opf` (Calibre), `metadata.json` (Audiobookshelf/Libation, array-or-string tolerant) | title (+subtitle join), author, series + index, narrators, abridged, ASIN, tags/genres, dates, publisher, description |
|
|
| 2 | **Folder structure** — `Author/Title`, `Author/Series, Book #N - Title`, `Author/Series/Title` | title, author, series + index |
|
|
| 3 | **Embedded tags** (via ffprobe) | `album` → title; `album_artist` → author (`artist` fallback); `composer` → **narrators** (Audible's own convention); `grouping` → series; `date` → publish date; `genre` → genre; Audible freeform atoms `----:com.apple.iTunes:SERIES`/`:SERIES-PART`/`:SUBTITLE`/`:ASIN`/`:ABRIDGED` |
|
|
| 4 | **Filename/folder name** | title last of all (created only — never lets a rescan blank a title) |
|
|
|
|
Abridged is the one guarded field: an explicit sidecar `false` wins even
|
|
when a tag says `true` (the sidecar's presence is tracked, not just its
|
|
value). Narrators from the composer tag stay ONE string — composer
|
|
values have no reliable multi-narrator delimiter, and splitting
|
|
"Vance, Simon" would manufacture fake people.
|
|
|
|
## Chapters (the `{index, title, start, end}` array)
|
|
|
|
| Rank | Source |
|
|
|---|---|
|
|
| 1 | Sidecar `metadata.json` `chapters[]` (curated, global seconds) |
|
|
| 2 | Embedded m4b chapter atoms (`chpl` / QuickTime text track, read by ffprobe) |
|
|
| 3 | **CUE sheet** — `.cue` next to the audio file (FILE-directive matched, `INDEX 01 MM:SS:FF` at 75 fps); the classic one-big-flac + cue rip shape |
|
|
| 4 | Track boundaries — for folder/zip books, each file is a chapter |
|
|
|
|
## Track ordering (multi-file books)
|
|
|
|
| Rank | Source |
|
|
|---|---|
|
|
| 1 | **Playlist** — exactly one `.m3u`/`.m3u8` among the book root's direct children; its entry order wins and `#EXTINF` names the tracks |
|
|
| 2 | **Tags** — `(disc, track)` sort when *every* track carries a track number (fixes flat `01.mp3` dumps from multi-disc rips) |
|
|
| 3 | **Natural sort** of relative paths (`02` before `10`), recursive — the default |
|
|
|
|
Track display titles, highest first: `movement_name` → `title` tag →
|
|
playlist `#EXTINF` → cleaned filename (leading ordinals/separators
|
|
stripped).
|
|
|
|
## Book grouping (which files form one book)
|
|
|
|
- The book root is the nearest ancestor holding audio files DIRECTLY,
|
|
merged upward through disc-part directories (`CD1`, `Disc 2`, `Part 3`,
|
|
`05`) when the parent has no audio of its own.
|
|
- Sibling directories are NEVER merged (two books under one author stay
|
|
two books); a root with a single track falls back to the single-file
|
|
model.
|
|
- A lone `.zip` of audio files is its own book (layout "zip"); a folder
|
|
of tracks is one book with `file_path` = the directory (layout
|
|
"folder"); a single audio file is the Batch A model (no layout key).
|
|
- Rescans take a fast path (track count + total size + known durations
|
|
unchanged → no re-probe) and absorb pre-grouping one-track strays.
|
|
|
|
## Covers
|
|
|
|
| Rank | Source |
|
|
|---|---|
|
|
| 1 | Sidecar `cover.jpg` / `folder.jpg` / … next to the book |
|
|
| 2 | **Embedded art** (`APIC` mp3, `covr` m4b, flac picture) — one optional ffmpeg call, written as a `<book>.cover.jpg` sibling following the EPUB convention |
|
|
|
|
## Accepted containers (audiobook libraries)
|
|
|
|
`.m4b` `.m4a` `.mp3` `.flac` `.zip` (book container) `.opus` `.ogg`
|
|
`.oga` `.aac` `.wav` — all browser-playable; served with their real
|
|
media types (`audio/ogg` per RFC 7845 for Opus-in-Ogg, etc.).
|
|
|
|
## Documented non-goals
|
|
|
|
- **ID3v2 CHAP/CTOC chapter frames** — the written MP3 standard for
|
|
embedded chapters, near-zero audiobook adoption; would need a
|
|
hand-rolled ID3v2.4 parser. Revisit only if real files show up.
|
|
- **NFO sidecars** — no genuine audiobook NFO standard exists (Jellyfin's
|
|
Bookshelf uses Calibre OPF, which we read).
|
|
- **ReplayGain / EBU R128 loudness tags** — a playback feature, not
|
|
ingestion; separate plan if ever wanted.
|
|
- **`.wma`** — browsers cannot play it; excluded deliberately.
|