John O'Keefe
7e4c97f639
Phase 2a of the metadata overhaul (plan §2.1-2.3, §2.5-2.9). The headline bug: ABS writes sidecar series as an array of STRINGS (["The Witcher #3", ...]); the parser only accepted {series, sequence} OBJECT arrays, so every ABS-matched book silently lost its series (found live: 9 sidecars, zero series ingested; only folder-structure Malazan survived). parseSeriesCandidates now normalizes string arrays, object arrays and Libation plain strings. Three-way series assignment (plan §2.2): - primary bibliographic series -> series/series_number columns - additional real series (Drizzt: Icewind Dale #1 = Legend of Drizzt #4) -> alternate_info JSONB, comic-established shape - ordering constructs ('The Witcher (Reading Order)') -> PlaylistSeries candidates; NEVER the series column. Detection is explicit-marker-only ((reading order|chronological|publication order|in order|timeline) as parenthetical/suffix) — precision over recall; a series merely containing 'timeline' mid-name stays a series. Sequence policy: leading-number extraction ('Book 2' -> 2, '2, Dramatized Adaptation' -> 2, '1a' -> 1); lossy extractions preserve the raw text in alternate_info.series_sequence_raw. Smart genre (deterministic lexicon): compound Audible blobs split ('Epic, Action & Adventure, Military, Sword & Sorcery' -> 4 candidates), alias-normalized, format noise demoted ('Audiobook' never wins), most-specific canonical genre takes the genre column, all candidates stay in tags. Language: display names normalize to ISO 639-1 on ingest; ROOT-CAUSED the update-path drop — CreateMediaItem call site never set Language (explicit NULL over the column default), the update path did; create now carries it. Title-derived series fallback (§2.7): 'Baptism of Fire: The Witcher, Book 3' 2-level folders split conservatively (only when the post-colon tail matches ', Book N'), subtitle parsed for series+number, blanks-only precedence, matching-name number fill, zero display mutation. Works with no sidecar and no network. Ingest fixes: subtitle/explicit land in their columns (audio tag subtitle stops joining into title); full authors[] array (was [0]- only); sidecar descriptions sanitized via bluemonday UGC policy (vendored). metadata_overrides: verified scan-update path already applies ApplyMetadataOverrides; subtitle/explicit/authors covered by the Phase 1 override adaptation. Tests: new series_parsing_test.go (string-array sidecar e2e, Drizzt alternate_info, lossy raw preservation, playlist classification, title-derived fallback incl. no-mutation + blanks-only, genre lexicon, language map); audio-tag subtitle pin updated to column behavior; calibre/ABS 2.5-series_index pins widened per float change (changelog'd in phase 1). Deferred to 2b: auto-created playlist collections (§2.4) — settings toggle, collection upsert, ownership.
143 lines
5.6 KiB
Go
143 lines
5.6 KiB
Go
package services
|
|
|
|
import "strings"
|
|
|
|
// Smart genre selection for sidecar/embedded metadata (A2c plan §2.5).
|
|
//
|
|
// Sources arrive as comma-blobs ("Epic, Action & Adventure, Military,
|
|
// Sword & Sorcery" — one Audible genre string), media-format noise
|
|
// ("Audiobook"), or maximally generic tags ("Fiction"). This is a
|
|
// deterministic ranked rule set — no network, no guessing service:
|
|
//
|
|
// 1. split candidate strings on , ; /
|
|
// 2. alias-map to canonical genres
|
|
// 3. discard demoted (format/edition) tokens — they never win
|
|
// 4. most specific (longest) canonical hit wins; "Fiction" is last resort
|
|
// 5. every candidate (split parts included) still lands in tags, so
|
|
// filtering keeps full coverage even when the genre column holds one
|
|
|
|
var genreAliases = map[string]string{
|
|
"sci-fi": "science fiction", "scifi": "science fiction", "sf": "science fiction",
|
|
"sff": "science fiction", "sci fi": "science fiction",
|
|
"speculative fiction": "science fiction",
|
|
" detectives": "mystery", "detective": "mystery",
|
|
"whodunnit": "mystery", "whodunit": "mystery",
|
|
"adventure fiction": "action & adventure",
|
|
"action and adventure": "action & adventure",
|
|
"ya": "young adult",
|
|
"mg": "middle grade",
|
|
"graphic novel": "graphic novels",
|
|
"coming of age": "coming of age",
|
|
"coming-of-age": "coming of age",
|
|
"histfiction": "historical fiction",
|
|
"epic": "epic fantasy",
|
|
"high fantasy": "epic fantasy",
|
|
"military scifi": "military science fiction",
|
|
"military sf": "military science fiction",
|
|
"space opera": "space opera",
|
|
"post apocalyptic": "post-apocalyptic",
|
|
"self help": "self-help",
|
|
"sci_fi": "science fiction",
|
|
}
|
|
|
|
// canonicalGenres is the known-genre set (lowercase). A candidate that
|
|
// normalizes to one of these wins the `genre` column; longest match beats
|
|
// shorter (Sword & Sorcery > Fiction).
|
|
var canonicalGenres = map[string]bool{
|
|
"fiction": true, "nonfiction": true, "literary fiction": true,
|
|
"classics": true, "poetry": true, "short stories": true,
|
|
"anthology": true, "coming of age": true, "satire": true, "humor": true,
|
|
"fantasy": true, "epic fantasy": true, "sword & sorcery": true,
|
|
"dark fantasy": true, "urban fantasy": true, "paranormal": true,
|
|
"paranormal romance": true, "high fantasy": true, "fairy tales": true,
|
|
"mythology": true, "folklore": true,
|
|
"science fiction": true, "space opera": true, "cyberpunk": true,
|
|
"military science fiction": true, "dystopian": true,
|
|
"post-apocalyptic": true, "apocalyptic": true, "time travel": true,
|
|
"alternate history": true, "first contact": true, "steampunk": true,
|
|
"litrpg": true, "progression fantasy": true,
|
|
"historical fiction": true, "alternate world": true,
|
|
"romance": true, "contemporary romance": true, "historical romance": true,
|
|
"mystery": true, "cozy mystery": true, "crime": true, "true crime": true,
|
|
"police procedural": true, "noir": true, "hardboiled": true,
|
|
"thriller": true, "suspense": true, "espionage": true, "horror": true,
|
|
"gothic": true, "supernatural": true,
|
|
"action & adventure": true, "adventure": true, "western": true,
|
|
"military": true, "war": true,
|
|
"biography": true, "autobiography": true, "memoir": true,
|
|
"history": true, "science": true, "philosophy": true,
|
|
"psychology": true, "self-help": true, "business": true,
|
|
"economics": true, "politics": true, "religion": true,
|
|
"spirituality": true, "travel": true, "cooking": true, "food": true,
|
|
"health": true, "fitness": true, "sports": true, "music": true,
|
|
"art": true, "photography": true, "education": true, "reference": true,
|
|
"young adult": true, "middle grade": true, "children's": true,
|
|
"comics": true, "graphic novels": true, "manga": true,
|
|
"legal": true, "medical": true,
|
|
}
|
|
|
|
// genreDemoted never wins the genre column — media formats and editions,
|
|
// not genres. They still pass through to tags.
|
|
var genreDemoted = map[string]bool{
|
|
"audiobook": true, "unabridged": true, "abridged": true,
|
|
"dramatization": true, "dramatized": true, "audio drama": true,
|
|
}
|
|
|
|
// splitGenreCandidates splits compound genre strings ("Epic, Action &
|
|
// Adventure" — one Audible blob) into individual candidates.
|
|
func splitGenreCandidates(genres []string) []string {
|
|
var out []string
|
|
for _, g := range genres {
|
|
for _, part := range strings.Split(g, ",") {
|
|
for _, p := range strings.Split(part, ";") {
|
|
for _, piece := range strings.Split(p, "/") {
|
|
t := strings.TrimSpace(piece)
|
|
if t != "" {
|
|
out = append(out, t)
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
return out
|
|
}
|
|
|
|
// pickSmartGenre chooses the best genre from candidate strings. Returns the
|
|
// winner ("" when there are no candidates) and the full candidate list for
|
|
// the tags array.
|
|
func pickSmartGenre(genres []string) (genre string, all []string) {
|
|
all = splitGenreCandidates(genres)
|
|
best := ""
|
|
bestLen := -1
|
|
for _, c := range all {
|
|
lower := strings.ToLower(strings.TrimSpace(c))
|
|
if lower == "" || genreDemoted[lower] {
|
|
continue
|
|
}
|
|
if genreAliases[lower] != "" {
|
|
lower = genreAliases[lower]
|
|
}
|
|
if !canonicalGenres[lower] {
|
|
continue
|
|
}
|
|
// Specificity: the longest canonical form wins; earlier candidates
|
|
// win ties (source order is provider significance).
|
|
if len(lower) > bestLen {
|
|
best = lower
|
|
bestLen = len(lower)
|
|
}
|
|
}
|
|
if best != "" {
|
|
return best, all
|
|
}
|
|
// No lexicon hit: fall back to the first non-demoted candidate verbatim
|
|
// (display honesty over false taxonomy), "" when only noise existed.
|
|
for _, c := range all {
|
|
lower := strings.ToLower(strings.TrimSpace(c))
|
|
if lower != "" && !genreDemoted[lower] {
|
|
return strings.TrimSpace(c), all
|
|
}
|
|
}
|
|
return "", all
|
|
}
|