Files

T

john-okeefe 008706a7fa docs: update implementation plan with DOUBLE PRECISION and complete integration tests

Phase 6.2 implementation: Add comprehensive integration test code to plan.

Documentation Updates:
- Changed community_rating from DECIMAL(3,1) to DOUBLE PRECISION throughout plan
- Fixed function name references: processNewMediaItems → processMediaFile (correct name)
- Added complete integration test implementation (TestComicMetadataExtraction,
  TestReadingDirectionAPI, TestUniversalMetadataFields, TestComicSpecificFields)
- Replaced skeleton TestMergeMetadata with actual test code using setupTestServer
- Added context-based location markers (3 lines before/after) for easier code navigation
- Removed TODO comment reference (doesn't exist in current code)
- Updated all line number references and added plan update summary
- Verified test helper usage: setupDeviceTest for library creation

Integration Tests Added:
- Full comic metadata field testing (manga_type, reading_direction, series_count, volume,
  imprint, age_rating, community_rating as pgtype.Float8, story_arc, is_black_and_white)
- Reading direction API testing (rtl, ltr, auto filtering)
- Universal field testing for both comic and ebook libraries
- Comic-specific field testing (alternate_info JSONB, scan_information, summary, metadata_notes)

All tests use proper test_helpers pattern with setupDeviceTest and verify DOUBLE PRECISION
storage for CommunityRating field.

Relates to: Phase 6.2 integration testing documentation

2026-03-29 21:12:27 -04:00

98 KiB

Raw Blame History

Implementation Plan: Smart Metadata Merge & Complete ComicInfo.xml Support

Scope: Extract ALL 19 ComicInfo.xml v2.0 fields (not just reading direction)

Status: Updated - Full ComicInfo.xml support (March 29, 2026)

⚠️ PLAN UPDATES (March 29, 2026)

This implementation plan has been corrected with the following changes:

Context-based location markers: All line number references have been replaced with 3-line context markers (before/after) to make locations easier to find even after code insertions shift line numbers.
Phase 1 (Database Schema): Marked as mostly complete. The schema columns, indexes, and comments are already in place. Only the SQL INSERT query needs updating.
Phase 4.1 (SQL Query): Added the complete SQL query code showing the 14 new columns and parameters.
Phase 4.4 (TODO Comment): Removed - this TODO comment doesn't exist in the current codebase.
Phase 2.2 (MediaMetadata): Updated with exact location of the struct (starts around line 42, not line 101).
Phase 3.4 (extractMetadata): Updated with context-based location and clarified that the fallback switch statement (.epub, .pdf) is removed as this logic moves into mergeMetadata().
Phase 4.4 (Duplicate Code): Renumbered from 4.5 and added context-based location markers.
Fixed function name: Corrected processNewMediaItems() → processMediaFile() throughout the plan. This is the actual function name in media_scanner.go.
Changed community_rating to DOUBLE PRECISION: Changed from DECIMAL(3,1) to DOUBLE PRECISION for simpler code. This eliminates the awkward pgtype.Numeric conversion and uses simple pgtype.Float8 instead. Matches ComicInfo.xml float64 type naturally with no string conversion needed. The floating-point precision error is negligible for 0-10 ratings (< 0.00001%).

What's Already Done:

Database schema columns (lines 137-161 in schema.sql)
Database indexes (lines 424-461 in schema.sql)
Go models regenerated (MediaItems struct lines 229-256 in models.go)
CreateMediaItemParams struct (lines 586-599 in queries.sql.go)

What Still Needs Doing:

Update CreateMediaItem SQL query in queries.sql
Update ComicInfo struct with new fields
Update MediaMetadata struct with new fields
Create mergeMetadata() function
Update extractMetadata() function
Update CreateMediaItem() call with new parameters
Remove duplicate comic extraction code
Update TypeScript types
Write tests
Write documentation

Overview

Add intelligent metadata merging for ALL media types (ebooks, comics, etc.) with comprehensive support for ALL ComicInfo.xml metadata fields (19 total fields). This feature will:

Parse ComicInfo.xml from comic archives (.cbz, .cbr, .cb7, .cbt)
Extract ALL metadata fields from ComicInfo.xml (reading direction, series info, imprint, story arc, age rating, etc.)
Implement smart metadata merging: Calibre metadata.opf + embedded metadata
Store both raw manga_type and computed reading_direction in database
Store universal fields that apply to ALL formats: age_rating, series_count, volume, imprint, web_url
Store comic-specific fields: story_arc, scan_information, alternate_info (JSONB), metadata_notes, community_rating
Implement genre/tag processing: ALL genres appear in tags array without duplication
Propagate all metadata through all layers: database → API → frontend

Project Context

Current State:

✅ Parses Calibre metadata.opf sidecar files (priority)
✅ Parses ComicInfo.xml from comic archives (fallback only)
✅ Extracts basic comic metadata (title, series, number, publisher, writer)
❌ Missing: 13 additional ComicInfo.xml fields not extracted (series_count, volume, imprint, story_arc, age_rating, etc.)
❌ Missing: Universal fields that apply to all formats (age_rating, series_count, web_url, imprint, volume)
❌ Missing: Manga field in ComicInfo struct
❌ Missing: Reading direction in database schema
❌ Missing: Comic-specific fields (volume, series count, imprint, story arc, age rating, etc.)
❌ Missing: Smart metadata merging (currently: if metadata.opf exists, skip ComicInfo.xml)
❌ Missing: Reading direction in API responses
❌ Missing: Reading direction in frontend types

Phase 1: Database Schema Changes

1.1 Add Columns to `media_items` Table

File: database/schema/schema.sql

Location: After line 133 (after kobo_metadata JSONB)

Status: ✅ Already completed - All columns, indexes, and comments are in place

Note: The schema has already been updated with all 14 new columns. The only change needed is to ensure community_rating is DOUBLE PRECISION (which it already is in the current schema).

Existing Schema (already in place):

-- Manga and comic reading direction support
-- Stores raw Manga field from ComicInfo.xml
manga_type VARCHAR(30) DEFAULT 'unknown'
CHECK (manga_type IN ('unknown', 'no', 'yes', 'yes_and_right_to_left')),

-- Stores computed reading direction for queries/UI
reading_direction VARCHAR(20) DEFAULT 'auto'
CHECK (reading_direction IN ('auto', 'ltr', 'rtl', 'vertical')),

-- Universal series information (applies to ALL formats: ebooks, audiobooks, comics)
series_count INTEGER,             -- Total items in series (from ComicInfo Count field, or book series count)
volume INTEGER,                   -- Volume/omnibus number for collected editions

-- Universal publisher and classification (applies to ALL formats)
imprint VARCHAR(255),             -- Publisher imprint (e.g., Vertigo, HarperCollinsEpic)
age_rating VARCHAR(20),           -- Age rating: Everyone, Teen, Mature, Adult (applies to all formats)
web_url VARCHAR(500),             -- URL to info page (Goodreads, ComicVine, MangaUpdates, Audible, etc.)

-- Comic-specific fields
story_arc VARCHAR(255),           -- Story arc name (e.g., "The Dark Phoenix Saga", "Civil War")
is_black_and_white BOOLEAN,       -- Black and white flag (mostly comics, some illustrated books)

-- Additional metadata (applies to all formats)
metadata_notes TEXT,              -- Notes from metadata files (ComicInfo.xml, EPUB, PDF) - distinct from user notes
community_rating DOUBLE PRECISION,    -- Pre-existing community rating from metadata (0.0-10.0) - distinct from user ratings

-- Alternate series information (JSONB for flexible schema - comic-specific)
alternate_info JSONB,             -- Stores AlternateSeries, AlternateNumber, AlternateCount
-- Example: {"alternate_series": "Ultimate X-Men", "alternate_number": 1, "alternate_count": 12}

-- Scan and publication metadata (comic-specific)
scan_information TEXT,            -- Scan information (scanner group, resolution, etc.)

-- Summary (distinct from description - may merge with Calibre description)
summary TEXT,                     -- Summary from ComicInfo.xml (may be merged with description from Calibre)

Summary of Phase 1:

✅ All 14 columns already added to schema
✅ All 8 indexes already created
✅ All column comments already added
✅ community_rating is DOUBLE PRECISION (not DECIMAL)
⚠️ Action needed: Only if you haven't regenerated sqlc models yet

1.2 Add Indexes for Comic and Universal Metadata Queries

Note: The indexes for these fields have already been created in the schema. This section is for reference only.

File: database/schema/schema.sql

Location: Find the section with existing comic metadata indexes (already implemented)

Existing Indexes (Already in schema.sql):

-- Index for filtering by reading direction (for manga/comic libraries)
CREATE INDEX IF NOT EXISTS idx_media_items_reading_direction
ON media_items(reading_direction)
WHERE reading_direction IS NOT NULL;

-- Index for filtering by story arc (comic-specific)
CREATE INDEX IF NOT EXISTS idx_media_items_story_arc
ON media_items(story_arc)
WHERE story_arc IS NOT NULL;

-- Index for filtering by imprint (universal - all formats)
CREATE INDEX IF NOT EXISTS idx_media_items_imprint
ON media_items(imprint)
WHERE imprint IS NOT NULL;

-- Index for filtering by age rating (universal - all formats)
CREATE INDEX IF NOT EXISTS idx_media_items_age_rating
ON media_items(age_rating)
WHERE age_rating IS NOT NULL;

-- Index for filtering by manga type (comic-specific)
CREATE INDEX IF NOT EXISTS idx_media_items_manga_type
ON media_items(manga_type)
WHERE manga_type IS NOT NULL;

-- Index for filtering by series count (universal - all formats)
CREATE INDEX IF NOT EXISTS idx_media_items_series_count
ON media_items(series_count)
WHERE series_count IS NOT NULL;

-- Index for filtering by volume (universal - all formats)
CREATE INDEX IF NOT EXISTS idx_media_items_volume
ON media_items(volume)
WHERE volume IS NOT NULL;

-- GIN index for alternate_info JSONB queries (comic-specific)
CREATE INDEX IF NOT EXISTS idx_media_items_alternate_info_gin
ON media_items USING GIN (alternate_info)
WHERE alternate_info IS NOT NULL;

Note: These indexes are already in the schema and do not need to be added again.

1.3 Add Column Comments

File: database/schema/schema.sql

Location: After line 478 (after existing column comments)

Changes:

-- Comic-specific fields
COMMENT ON COLUMN media_items.manga_type IS 'Raw Manga field from ComicInfo.xml: unknown, no, yes, yes_and_right_to_left';
COMMENT ON COLUMN media_items.reading_direction IS 'Computed reading direction: auto, ltr (left-to-right), rtl (right-to-left), vertical (webtoons/manhwa)';
COMMENT ON COLUMN media_items.story_arc IS 'Story arc name for grouping related issues (e.g., "The Dark Phoenix Saga", "Civil War")';
COMMENT ON COLUMN media_items.is_black_and_white IS 'Black and white comic flag from ComicInfo.xml';
COMMENT ON COLUMN media_items.alternate_info IS 'Alternate series information as JSONB: {alternate_series, alternate_number, alternate_count}';
COMMENT ON COLUMN media_items.scan_information IS 'Scan information from ComicInfo.xml (scanner group, resolution, etc.)';

-- Universal fields (apply to ebooks, audiobooks, comics)
COMMENT ON COLUMN media_items.series_count IS 'Total items in series (from ComicInfo.xml Count field, or book series count)';
COMMENT ON COLUMN media_items.volume IS 'Volume/omnibus number for collected editions';
COMMENT ON COLUMN media_items.imprint IS 'Publisher imprint/subdivision (e.g., Vertigo, HarperCollinsEpic, DC Black Label)';
COMMENT ON COLUMN media_items.age_rating IS 'Age rating from metadata: Everyone, Teen, Mature, Adult (applies to all formats)';
COMMENT ON COLUMN media_items.web_url IS 'URL to info page (Goodreads, ComicVine, MangaUpdates, Audible, etc.)';

-- Additional metadata (applies to all formats)
COMMENT ON COLUMN media_items.metadata_notes IS 'Notes from metadata files (ComicInfo.xml, EPUB, PDF) - distinct from user notes in media_notes table';
COMMENT ON COLUMN media_items.community_rating IS 'Pre-existing community rating from metadata files (scale 0.0-10.0, DOUBLE PRECISION) - distinct from user ratings in media_ratings table';
COMMENT ON COLUMN media_items.summary IS 'Summary from ComicInfo.xml (may be merged with description from Calibre)';

1.4 Regenerate Database Models

Command:

cd internal/database && sqlc generate

Expected Changes:

CreateMediaItemParams struct will include MangaType and ReadingDirection fields
UpdateMediaItemIdentifiersParams struct may need updating
All queries that return MediaItems will include new columns

Phase 2: Update Data Structures

2.1 Enhance `ComicInfo` Struct

File: internal/services/media_scanner.go

Location: Find the struct starting with:

// ComicInfo represents metadata from ComicInfo.xml
type ComicInfo struct {

Current Code:

// ComicInfo represents metadata from ComicInfo.xml
type ComicInfo struct {
    XMLName     xml.Name `xml:"ComicInfo"`
    Title       string   `xml:"Title"`
    Series      string   `xml:"Series"`
    Number      int      `xml:"Number"`
    Volume      int      `xml:"Volume"`
    Publisher   string   `xml:"Publisher"`
    Year        int      `xml:"Year"`
    Month       int      `xml:"Month"`
    Day         int      `xml:"Day"`
    Writer      string   `xml:"Writer"`
    Penciller   string   `xml:"Penciller"`
    Inker       string   `xml:"Inker"`
    Colorist    string   `xml:"Colorist"`
    Letterer    string   `xml:"Letterer"`
    CoverArtist string   `xml:"CoverArtist"`
    Genre       string   `xml:"Genre"`
    Tags        string   `xml:"Tags"`
    Web         string   `xml:"Web"`
    Notes       string   `xml:"Notes"`
}

Updated Code:

type ComicInfo struct {
    XMLName        xml.Name `xml:"ComicInfo"`

    // Basic metadata (already extracted)
    Title          string   `xml:"Title"`
    Series         string   `xml:"Series"`
    Number         int      `xml:"Number"`
    Volume         int      `xml:"Volume"`
    Publisher      string   `xml:"Publisher"`
    Year           int      `xml:"Year"`
    Month          int      `xml:"Month"`
    Day            int      `xml:"Day"`
    Writer         string   `xml:"Writer"`
    Penciller      string   `xml:"Penciller"`
    Inker          string   `xml:"Inker"`
    Colorist       string   `xml:"Colorist"`
    Letterer       string   `xml:"Letterer"`
    CoverArtist    string   `xml:"CoverArtist"`
    Genre          string   `xml:"Genre"`
    Tags           string   `xml:"Tags"`
    Web            string   `xml:"Web"`
    Notes          string   `xml:"Notes"`

    // NEW: Reading direction fields from ComicInfo.xml v2.0
    Manga          string   `xml:"Manga"`       // Unknown, No, Yes, YesAndRightToLeft
    LanguageISO    string   `xml:"LanguageISO"` // ISO 639-1 language code for heuristics

    // NEW: Additional comic-specific fields (19 total fields from ComicInfo.xml)
    Count          int      `xml:"Count"`          // Total issues in series
    AlternateSeries string   `xml:"AlternateSeries"`
    AlternateNumber int      `xml:"AlternateNumber"`
    AlternateCount  int      `xml:"AlternateCount"`
    Summary        string   `xml:"Summary"`
    Imprint        string   `xml:"Imprint"`
    StoryArc       string   `xml:"StoryArc"`
    SeriesGroup    string   `xml:"SeriesGroup"`
    AgeRating      string   `xml:"AgeRating"`
    CommunityRating float64  `xml:"CommunityRating"`
    MainCharacterOrTeam string `xml:"MainCharacterOrTeam"`
    Review         string   `xml:"Review"`
    BlackAndWhite  string   `xml:"BlackAndWhite"` // "Yes" or "No"
    ScanInformation string  `xml:"ScanInformation"`
    Characters      string   `xml:"Characters"`
    Teams          string   `xml:"Teams"`
    Locations       string   `xml:"Locations"`
}

2.2 Enhance `MediaMetadata` Struct

File: internal/services/media_scanner.go

Location: Find the struct starting with:

// MediaMetadata contains extracted metadata for media files (ebooks, comics, manga)
type MediaMetadata struct {

Current Code:

// MediaMetadata contains extracted metadata for media files (ebooks, comics, manga)
type MediaMetadata struct {
    Title        string
    Author       string
    Description  string
    Series       string
    SeriesNumber int32
    Publisher    string
    PublishDate  time.Time
    Contributors []string
    CoverPath    string
    ISBN         string
    ASIN         string
    Tags         []string

    FileHashInfo *HashInfo
    FileFormats  []*FormatInfo
}

Updated Code:

// MediaMetadata contains extracted metadata for media files (ebooks, comics, manga)
type MediaMetadata struct {
    // Existing fields
    Title        string
    Author       string
    Description  string
    Series       string
    SeriesNumber int32
    Publisher    string
    PublishDate  time.Time
    Contributors []string
    CoverPath    string
    ISBN         string
    ASIN         string
    Tags         []string

    FileHashInfo *HashInfo
    FileFormats  []*FormatInfo

    // NEW: Reading direction fields (from ComicInfo.xml or computed)
    MangaType        string  // Raw ComicInfo.xml Manga field
    ReadingDirection string  // Computed: auto, ltr, rtl, vertical
    Language         string  // ISO 639-1 language code

    // NEW: Additional metadata fields (from ComicInfo.xml or other metadata sources)
    // Universal fields (apply to ebooks, audiobooks, comics)
    SeriesCount      int32   // Total items in series (Count field for comics, series count for books)
    Volume           int32   // Volume/omnibus number
    Imprint          string  // Publisher imprint
    AgeRating        string  // Age rating (Everyone, Teen, Mature, Adult)
    WebURL           string  // URL to info page (Goodreads, ComicVine, etc.)
    MetadataNotes    string  // Notes from metadata files (not user notes)
    CommunityRating  float64 // Pre-existing community rating (0.0-10.0) - maps to DOUBLE PRECISION in database

    // Comic-specific fields
    StoryArc         string  // Story arc name
    IsBlackAndWhite  bool    // Black and white flag
    AlternateInfo    string  // JSONB string of alternate series info
    ScanInformation  string  // Scan information
    Summary          string  // Summary from ComicInfo.xml
}

Phase 3: Smart Metadata Merging Logic

3.1 Create Metadata Merge Function

File: internal/services/media_scanner.go

Location: Insert this new function after extractCalibreSidecar() function. Find the function ending with:

    return metadata
}

and before the next function. The new mergeMetadata() function should be inserted here.

New Function:

// mergeMetadata intelligently merges metadata from multiple sources
// Priority: metadata.opf (Calibre) → embedded metadata → folder structure → filename
// For comics: metadata.opf → ComicInfo.xml → folder structure → filename
func (s *MediaScanner) mergeMetadata(path string, calibreMetadata *MediaMetadata) (*MediaMetadata, error) {
    metadata := calibreMetadata
    if metadata == nil {
        metadata = &MediaMetadata{}
    }

    ext := strings.ToLower(filepath.Ext(path))

    // For comic archives, try to extract ComicInfo.xml
    if ext == ".cbz" || ext == ".cbr" || ext == ".cb7" || ext == ".cbt" {
        comicInfo, cover, err := extractComicMetadata(path)
        if err != nil {
            fmt.Printf("Warning: failed to extract comic metadata from %s: %v\n", path, err)
        } else if comicInfo != nil {
            // Merge ComicInfo.xml fields (only if not already set from Calibre)
            if metadata.Title == "" && comicInfo.Title != "" {
                metadata.Title = comicInfo.Title
            }
            if metadata.Series == "" && comicInfo.Series != "" {
                metadata.Series = comicInfo.Series
            }
            if metadata.SeriesNumber == 0 && comicInfo.Number > 0 {
                metadata.SeriesNumber = int32(comicInfo.Number)
            }
            if metadata.Publisher == "" && comicInfo.Publisher != "" {
                metadata.Publisher = comicInfo.Publisher
            }
            if metadata.Author == "" && comicInfo.Writer != "" {
                metadata.Author = comicInfo.Writer
            }
            if metadata.Description == "" && comicInfo.Summary != "" {
                metadata.Description = comicInfo.Summary
            }

            // NEW: Always extract reading direction from ComicInfo.xml
            // (even if metadata.opf exists, since Calibre doesn't support this field)
            metadata.MangaType = normalizeMangaType(comicInfo.Manga)
            metadata.ReadingDirection = determineReadingDirection(comicInfo)
            metadata.Language = comicInfo.LanguageISO

            // NEW: Extract additional comic-specific fields
            // Series information
            if metadata.SeriesCount == 0 && comicInfo.Count > 0 {
                metadata.SeriesCount = int32(comicInfo.Count)
            }
            if metadata.Volume == 0 && comicInfo.Volume > 0 {
                metadata.Volume = int32(comicInfo.Volume)
            }

            // Publisher and classification
            if metadata.Imprint == "" && comicInfo.Imprint != "" {
                metadata.Imprint = comicInfo.Imprint
            }
            if metadata.StoryArc == "" && comicInfo.StoryArc != "" {
                metadata.StoryArc = comicInfo.StoryArc
            }
            if metadata.AgeRating == "" && comicInfo.AgeRating != "" {
                metadata.AgeRating = normalizeAgeRating(comicInfo.AgeRating)
            }

            // NEW: Process genres and tags (universal logic for all formats)
            // Extract genre tags from ComicInfo.xml (Genre + Tags + Characters + Teams + Locations)
            genreTags := extractGenreTagsFromComicInfo(comicInfo)
            processGenresAndTags(metadata, genreTags)

            // Additional metadata
            if metadata.WebURL == "" && comicInfo.Web != "" {
                metadata.WebURL = comicInfo.Web
            }
            if metadata.MetadataNotes == "" && comicInfo.Notes != "" {
                metadata.MetadataNotes = comicInfo.Notes
            }
            if metadata.ScanInformation == "" && comicInfo.ScanInformation != "" {
                metadata.ScanInformation = comicInfo.ScanInformation
            }
            if metadata.Summary == "" && comicInfo.Summary != "" {
                metadata.Summary = comicInfo.Summary
            }

            // Boolean fields
            if !metadata.IsBlackAndWhite && strings.ToLower(comicInfo.BlackAndWhite) == "yes" {
                metadata.IsBlackAndWhite = true
            }
            if metadata.CommunityRating == 0 && comicInfo.CommunityRating > 0 {
                metadata.CommunityRating = comicInfo.CommunityRating
            }

            // Alternate series information (store as JSONB string)
            if metadata.AlternateInfo == "" && (comicInfo.AlternateSeries != "" || comicInfo.AlternateNumber > 0) {
                alternateData := map[string]interface{}{}
                if comicInfo.AlternateSeries != "" {
                    alternateData["alternate_series"] = comicInfo.AlternateSeries
                }
                if comicInfo.AlternateNumber > 0 {
                    alternateData["alternate_number"] = comicInfo.AlternateNumber
                }
                if comicInfo.AlternateCount > 0 {
                    alternateData["alternate_count"] = comicInfo.AlternateCount
                }
                if len(alternateData) > 0 {
                    jsonBytes, err := json.Marshal(alternateData)
                    if err == nil {
                        metadata.AlternateInfo = string(jsonBytes)
                    }
                }
            }

            // REMOVED: Tag enhancement now handled by processGenresAndTags()
            // Characters, Teams, Locations are already processed via extractGenreTagsFromComicInfo()

            // Extract cover if not already present
            if len(cover) > 0 && metadata.CoverPath == "" {
                coverPath := path + ".cover.jpg"
                if err := os.WriteFile(coverPath, cover, 0644); err == nil {
                    metadata.CoverPath = s.getRelativePath(coverPath)
                }
            }

            fmt.Printf("Merged comic metadata from %s: title=%s, series=%s, issue=%d, manga=%s, direction=%s\n",
                path, comicInfo.Title, comicInfo.Series, comicInfo.Number, comicInfo.Manga, metadata.ReadingDirection)
        }
    }

    return metadata, nil
}

// containsTag checks if a tag already exists in the tags array
func containsTag(tags []string, tag string) bool {
    tag = strings.ToLower(tag)
    for _, t := range tags {
        if strings.ToLower(t) == tag {
            return true
        }
    }
    return false
}

// normalizeAgeRating normalizes age rating from ComicInfo.xml to standard values
func normalizeAgeRating(rating string) string {
    rating = strings.ToLower(strings.TrimSpace(rating))
    switch rating {
    case "everyone", "e", "all ages":
        return "Everyone"
    case "teen", "t", "13+", "13+up":
        return "Teen"
    case "mature", "m", "17+", "17+up", "adults only":
        return "Mature"
    case "adult", "a", "18+":
        return "Adult"
    default:
        return rating // Return original if unknown
    }
}

3.2 Create Reading Direction Helper Functions

File: internal/services/media_scanner.go

Location: After mergeMetadata() function

New Functions:

// normalizeMangaType normalizes ComicInfo.xml Manga field to database enum values
func normalizeMangaType(manga string) string {
    switch strings.ToLower(strings.ReplaceAll(manga, " ", "")) {
    case "unknown":
        return "unknown"
    case "no":
        return "no"
    case "yes":
        return "yes"
    case "yesandrighttoleft":
        return "yes_and_right_to_left"
    default:
        return "unknown"
    }
}

// determineReadingDirection computes reading direction from ComicInfo metadata
// Uses Manga field + language heuristics + genre tags
func determineReadingDirection(comicInfo *ComicInfo) string {
    // 1. Check explicit Manga field
    manga := normalizeMangaType(comicInfo.Manga)
    switch manga {
    case "yes_and_right_to_left":
        return "rtl" // Traditional Japanese manga
    case "yes", "no":
        return "ltr" // Manga style but LTR, or Western comic
    }

    // 2. Language heuristic: Japanese → RTL
    lang := strings.ToLower(comicInfo.LanguageISO)
    if lang == "ja" || lang == "jpn" {
        return "rtl"
    }

    // 3. Genre heuristic: webtoons/manhwa → vertical
    tags := strings.ToLower(comicInfo.Tags + " " + comicInfo.Genre)
    if strings.Contains(tags, "webtoon") || strings.Contains(tags, "manhwa") {
        return "vertical" // Korean/Chinese webcomics
    }
    if strings.Contains(tags, "manga") && (lang == "ja" || lang == "jpn") {
        return "rtl" // Japanese manga
    }

    // 4. Default: LTR (Western comics)
    return "ltr"
}

3.3 Genre and Tag Processing Logic (Universal for ALL Formats)

File: internal/services/media_scanner.go

Location: After normalizeAgeRating() function

New Function:

// processGenresAndTags ensures ALL genres appear in the tags array without duplication
// This applies to ALL formats: EPUB, ComicInfo.xml, PDF metadata
// Strategy: Use existing `genre` column for primary genre, `tags` array for all genres
func processGenresAndTags(metadata *MediaMetadata, genreTags []string) {
    if metadata.Tags == nil {
        metadata.Tags = []string{}
    }

    // 1. Set primary genre (first genre tag wins if not already set)
    if metadata.Genre == "" && len(genreTags) > 0 {
        metadata.Genre = genreTags[0]
    }

    // 2. Ensure ALL genre tags appear in tags array (without duplication)
    for _, genreTag := range genreTags {
        genreTag = strings.TrimSpace(genreTag)
        if genreTag != "" && !containsTag(metadata.Tags, genreTag) {
            metadata.Tags = append(metadata.Tags, genreTag)
        }
    }
}

// extractGenreTagsFromEPUB extracts all <dc:subject> values from EPUB
// Returns array of genre tags
func extractGenreTagsFromEPUB(book *epub.Book) []string {
    var genreTags []string

    // EPUB stores genres in <dc:subject> metadata
    if subjects, err := book.MetadataByKey("subject"); err == nil && len(subjects) > 0 {
        for _, subject := range subjects {
            subject = strings.TrimSpace(subject)
            if subject != "" {
                genreTags = append(genreTags, subject)
            }
        }
    }

    return genreTags
}

// extractGenreTagsFromComicInfo extracts genres from ComicInfo.xml
// Genre field + Tags field + Characters + Teams + Locations
// Returns array of genre tags
func extractGenreTagsFromComicInfo(comicInfo *ComicInfo) []string {
    var genreTags []string

    // 1. Add Genre field
    if comicInfo.Genre != "" {
        genreTags = append(genreTags, strings.Split(comicInfo.Genre, ",")...)
    }

    // 2. Add Tags field (comma-separated)
    if comicInfo.Tags != "" {
        genreTags = append(genreTags, strings.Split(comicInfo.Tags, ",")...)
    }

    // 3. Add Characters (comma-separated)
    if comicInfo.Characters != "" {
        genreTags = append(genreTags, strings.Split(comicInfo.Characters, ",")...)
    }

    // 4. Add Teams (comma-separated)
    if comicInfo.Teams != "" {
        genreTags = append(genreTags, strings.Split(comicInfo.Teams, ",")...)
    }

    // 5. Add Locations (comma-separated)
    if comicInfo.Locations != "" {
        genreTags = append(genreTags, strings.Split(comicInfo.Locations, ",")...)
    }

    // Trim whitespace from all tags
    for i := range genreTags {
        genreTags[i] = strings.TrimSpace(genreTags[i])
    }

    return genreTags
}

Usage in mergeMetadata():

// In mergeMetadata() function, after extracting metadata:

// For EPUB files
if ext == ".epub" {
    book, err := epub.ReadBook(path)
    if err == nil {
        genreTags := extractGenreTagsFromEPUB(book)
        processGenresAndTags(metadata, genreTags)
    }
}

// For comic archives
if ext == ".cbz" || ext == ".cbr" || ext == ".cb7" || ext == ".cbt" {
    comicInfo, cover, err := extractComicMetadata(path)
    if err == nil && comicInfo != nil {
        genreTags := extractGenreTagsFromComicInfo(comicInfo)
        processGenresAndTags(metadata, genreTags)
    }
}

Key Points:

Single genre column: Primary classification (first genre tag)
tags array: ALL genres/tags without duplication
Universal logic: Works the same for EPUB, ComicInfo.xml, PDF metadata
No genre_comic column needed: Reuse existing genre column
Deduplication: containsTag() helper prevents duplicates

Examples:

EPUB:

Input: <dc:subject>Fantasy</dc:subject><dc:subject>Adventure</dc:subject>
Result: genre = "Fantasy", tags = ["Fantasy", "Adventure"]

ComicInfo.xml:

Input: <Genre>Action</Genre><Tags>Adventure, Fighting</Tags><Characters>Naruto, Sasuke</Characters>
Result: genre = "Action", tags = ["Action", "Adventure", "Fighting", "Naruto", "Sasuke"]

Calibre metadata.opf:

Input: Multiple <dc:subject> tags
Result: genre = first subject, tags = all subjects

3.4 Update `extractMetadata()` Function

File: internal/services/media_scanner.go

Location: Find the function starting with:

func (s *MediaScanner) extractMetadata(path string) (*MediaMetadata, error) {

Current Code (first ~15 lines of the function):

func (s *MediaScanner) extractMetadata(path string) (*MediaMetadata, error) {
    // NEW: Try Calibre sidecar first
    if metadata := s.extractCalibreSidecar(path); metadata != nil {
        fmt.Printf("Using Calibre metadata.opf for %s\n", path)

        // Try to find cover image for sidecar metadata
        coverPath := findSidecarCover(path)
        if coverPath != "" {
            metadata.CoverPath = s.getRelativePath(coverPath)
        }

        return metadata, nil
    }

    // EXISTING: Fallback to embedded metadata
    ext := strings.ToLower(filepath.Ext(path))
    ...

Updated Code:

func (s *MediaScanner) extractMetadata(path string) (*MediaMetadata, error) {
    // Try Calibre sidecar first
    calibreMetadata := s.extractCalibreSidecar(path)
    if calibreMetadata != nil {
        fmt.Printf("Using Calibre metadata.opf for %s\n", path)

        // Try to find cover image for sidecar metadata
        coverPath := findSidecarCover(path)
        if coverPath != "" {
            calibreMetadata.CoverPath = s.getRelativePath(coverPath)
        }
    }

    // NEW: Smart merge - also parse embedded metadata for missing fields
    // This ensures we get reading direction from ComicInfo.xml even when metadata.opf exists
    return s.mergeMetadata(path, calibreMetadata)
}

Key Changes:

Rename local variable from metadata to calibreMetadata for clarity
Remove early return when Calibre metadata exists
Call mergeMetadata() to merge Calibre + embedded metadata
Remove the fallback switch statement (.epub, .pdf, etc.) - this logic moves into mergeMetadata()

Phase 4: Update Media Item Creation

4.1 Update CreateMediaItem SQL Query

File: internal/database/queries/queries.sql

Location: Find the section starting with:

-- name: CreateMediaItem :one
INSERT INTO media_items (library_id, title, author, isbn...

and ending with:

...added_by_admin_id, created_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28)
RETURNING *;

Current Query:

-- name: CreateMediaItem :one
INSERT INTO media_items (library_id, title, author, isbn, description, file_path, file_size, mime_type, cover_image_path, series, series_number, tags, tags_search, asin, date_published, publisher, contributors, contributors_search, language, edition, page_count, genre, copyright_year, goodreads_id, openlibrary_id, google_books_id, added_by_admin_id, created_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28)
RETURNING *;

Updated Query:

-- name: CreateMediaItem :one
INSERT INTO media_items (library_id, title, author, isbn, description, file_path, file_size, mime_type, cover_image_path, series, series_number, tags, tags_search, asin, date_published, publisher, contributors, contributors_search, language, edition, page_count, genre, copyright_year, goodreads_id, openlibrary_id, google_books_id, added_by_admin_id, created_at, manga_type, reading_direction, series_count, volume, imprint, age_rating, web_url, story_arc, is_black_and_white, metadata_notes, community_rating, alternate_info, scan_information, summary)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35, $36, $37, $38, $39, $40, $41, $42)
RETURNING *;

Changes:

Added 14 new columns to INSERT: manga_type, reading_direction, series_count, volume, imprint, age_rating, web_url, story_arc, is_black_and_white, metadata_notes, community_rating, alternate_info, scan_information, summary
Added 14 new VALUES parameters: $29-$42 (note: count is 14 new parameters, not 15 as previously stated)

4.2 Regenerate sqlc Models

Command:

cd internal/database && sqlc generate

Expected Changes:

CreateMediaItemParams struct will include 14 new fields
MediaItems.CommunityRating will be pgtype.Float8 (not pgtype.Numeric)
All SELECT/RETURNING queries will include the new columns

Important: After running sqlc generate, verify that CommunityRating in the generated MediaItems struct is pgtype.Float8, not pgtype.Numeric.

4.3 Update CreateMediaItem Call

File: internal/services/media_scanner.go

Location: Find this section in processMediaFile() function:

    // Create media item in database
    relativePath := s.getRelativePath(path)
    createdItem, err := s.db.CreateMediaItem(ctx, database.CreateMediaItemParams{
        LibraryID:          libraryID,
        Title:              metadata.Title,
        Author:             pgtype.Text{String: metadata.Author, Valid: metadata.Author != ""},
        Isbn:               pgtype.Text{String: utils.NormalizeISBNSafe(metadata.ISBN), Valid: metadata.ISBN != ""},
        ...
        AddedByAdminID:     s.adminID,
        CreatedAt:          pgtype.Timestamptz{Time: fileModTime, Valid: true},
    })

Add to CreateMediaItem call (insert before the closing }):

    // Create media item in database
    relativePath := s.getRelativePath(path)
    createdItem, err := s.db.CreateMediaItem(ctx, database.CreateMediaItemParams{
        LibraryID:          libraryID,
        Title:              metadata.Title,
        Author:             pgtype.Text{String: metadata.Author, Valid: metadata.Author != ""},
        Isbn:               pgtype.Text{String: utils.NormalizeISBNSafe(metadata.ISBN), Valid: metadata.ISBN != ""},
        Asin:               pgtype.Text{String: metadata.ASIN, Valid: metadata.ASIN != ""},
        Description:        pgtype.Text{String: metadata.Description, Valid: metadata.Description != ""},
        FilePath:           relativePath,
        FileSize:           pgtype.Int8{Int64: info.Size(), Valid: true},
        MimeType:           pgtype.Text{String: s.getMimeType(path), Valid: true},
        CoverImagePath:     pgtype.Text{String: metadata.CoverPath, Valid: metadata.CoverPath != ""},
        Series:             pgtype.Text{String: metadata.Series, Valid: metadata.Series != ""},
        SeriesNumber:       pgtype.Int4{Int32: metadata.SeriesNumber, Valid: metadata.SeriesNumber > 0},
        Publisher:          pgtype.Text{String: metadata.Publisher, Valid: metadata.Publisher != ""},
        DatePublished:      pgtype.Date{Time: metadata.PublishDate, Valid: !metadata.PublishDate.IsZero()},
        Contributors:       metadata.Contributors,
        ContributorsSearch: contributorsSearch,
        Tags:               metadata.Tags,
        TagsSearch:         tagsSearch,
        AddedByAdminID:     s.adminID,
        CreatedAt:          pgtype.Timestamptz{Time: fileModTime, Valid: true},

        // NEW: Reading direction and comic metadata fields
        MangaType:        pgtype.Text{String: metadata.MangaType, Valid: metadata.MangaType != ""},
        ReadingDirection: pgtype.Text{String: metadata.ReadingDirection, Valid: metadata.ReadingDirection != ""},
        SeriesCount:      pgtype.Int4{Int32: metadata.SeriesCount, Valid: metadata.SeriesCount > 0},
        Volume:           pgtype.Int4{Int32: metadata.Volume, Valid: metadata.Volume > 0},
        Imprint:          pgtype.Text{String: metadata.Imprint, Valid: metadata.Imprint != ""},
        AgeRating:        pgtype.Text{String: metadata.AgeRating, Valid: metadata.AgeRating != ""},
        WebUrl:           pgtype.Text{String: metadata.WebURL, Valid: metadata.WebURL != ""},
        StoryArc:         pgtype.Text{String: metadata.StoryArc, Valid: metadata.StoryArc != ""},
        IsBlackAndWhite:  pgtype.Bool{Bool: metadata.IsBlackAndWhite, Valid: true},
        MetadataNotes:    pgtype.Text{String: metadata.MetadataNotes, Valid: metadata.MetadataNotes != ""},
        AlternateInfo:    func() []byte {
            if metadata.AlternateInfo != "" {
                return []byte(metadata.AlternateInfo)
            }
            return nil
        }(),
        ScanInformation:  pgtype.Text{String: metadata.ScanInformation, Valid: metadata.ScanInformation != ""},
        Summary:          pgtype.Text{String: metadata.Summary, Valid: metadata.Summary != ""},
        CommunityRating: pgtype.Float8{Float64: metadata.CommunityRating, Valid: metadata.CommunityRating > 0},
    })

Important Type Conversions:

AlternateInfo: string → []byte (JSONB in database)
CommunityRating: float64 (0-10 scale) → pgtype.Float8 (DOUBLE PRECISION) - simple direct assignment
WebUrl: Field name is WebUrl (camelCase) not WebURL

4.4 Remove Duplicate Comic Metadata Extraction

File: internal/services/media_scanner.go

Location: In processMediaFile() function, find this section:

    var comicInfo *ComicInfo
    var coverImage []byte

    ext := strings.ToLower(filepath.Ext(path))
    if ext == ".cbz" || ext == ".cbr" || ext == ".cb7" || ext == ".cbt" ||
        strings.HasSuffix(strings.ToLower(path), ".tar.gz") ||
        strings.HasSuffix(strings.ToLower(path), ".tar.bz2") ||
        strings.HasSuffix(strings.ToLower(path), ".tgz") ||
        strings.HasSuffix(strings.ToLower(path), ".tbz2") {
        info, cover, err := extractComicMetadata(path)
        if err != nil {
            fmt.Printf("Warning: failed to extract comic metadata from %s: %v\n", path, err)
        } else {
            comicInfo = info
            coverImage = cover
            if comicInfo.Title != "" && metadata.Title == "" {
                metadata.Title = comicInfo.Title
            }
            if comicInfo.Series != "" && metadata.Series == "" {
                metadata.Series = comicInfo.Series
            }
            if comicInfo.Number > 0 && metadata.SeriesNumber == 0 {
                metadata.SeriesNumber = int32(comicInfo.Number)
            }
            if comicInfo.Publisher != "" && metadata.Publisher == "" {
                metadata.Publisher = comicInfo.Publisher
            }
            if comicInfo.Writer != "" && metadata.Author == "" {
                metadata.Author = comicInfo.Writer
            }
            if len(coverImage) > 0 && metadata.CoverPath == "" {
                coverPath := path + ".cover.jpg"
                if err := os.WriteFile(coverPath, coverImage, 0644); err == nil {
                    metadata.CoverPath = s.getRelativePath(coverPath)
                }
            }
            fmt.Printf("Extracted comic metadata from %s: title=%s, series=%s, issue=%d\n",
                path, comicInfo.Title, comicInfo.Series, comicInfo.Number)
        }
    }

Remove this entire block and replace with:

    // REMOVED: Comic metadata extraction now handled by mergeMetadata()
    // This avoids duplicate extraction and ensures smart merging happens

Reason: This block was extracting comic metadata and merging fields manually. The new mergeMetadata() function handles this more comprehensively with all 29 ComicInfo.xml fields.

Phase 5: API Layer Updates

5.1 Update TypeScript API Types

File: web/src/types/api.d.ts

Location: Find the interface starting with:

interface MediaItemSummary {
    id: string;
    library_id: string;
    title: string;
    ...
    library_name: string;
    library_type_name: string;
}

Current Code:

interface MediaItemSummary {
  id: string;
  library_id: string;
  title: string;
  author?: string;
  isbn?: string;
  description?: string;
  file_path: string;
  file_size?: number;
  mime_type?: string;
  cover_image_path?: string;
  series?: string;
  series_number?: number;
  tags?: string[];
  asin?: string;
  date_published?: string;
  publisher?: string;
  contributors?: string[];
  language?: string;
  edition?: string;
  page_count?: number;
  genre?: string;
  copyright_year?: number;
  goodreads_id?: string;
  openlibrary_id?: string;
  google_books_id?: string;
  added_by_admin_id?: string;
  created_at: string;
  updated_at: string;
  format_group: string;
  format_mimetype?: string;
  is_reflowable?: boolean;
  has_fixed_layout?: boolean;
  total_characters?: number;
  chapter_count?: number;
  entitlement_id?: string;
  revision_number?: number;
  kobo_content_id?: string;
  kobo_metadata?: string;
  tags_search?: string[];
  contributors_search?: string[];
  file_sha256?: string;
  opf_identifier?: string;
  opf_uuid?: string;
  hash_confidence?: string;
  library_name: string;
  library_type_name: string;
}

Updated Code:

interface MediaItemSummary {
  id: string;
  library_id: string;
  title: string;
  author?: string;
  isbn?: string;
  description?: string;
  file_path: string;
  file_size?: number;
  mime_type?: string;
  cover_image_path?: string;
  series?: string;
  series_number?: number;
  tags?: string[];
  asin?: string;
  date_published?: string;
  publisher?: string;
  contributors?: string[];
  language?: string;
  edition?: string;
  page_count?: number;
  genre?: string;
  copyright_year?: number;
  goodreads_id?: string;
  openlibrary_id?: string;
  google_books_id?: string;
  added_by_admin_id?: string;
  created_at: string;
  updated_at: string;
  format_group: string;
  format_mimetype?: string;
  is_reflowable?: boolean;
  has_fixed_layout?: boolean;
  total_characters?: number;
  chapter_count?: number;
  entitlement_id?: string;
  revision_number?: number;
  kobo_content_id?: string;
  kobo_metadata?: string;
  tags_search?: string[];
  contributors_search?: string[];
  file_sha256?: string;
  opf_identifier?: string;
  opf_uuid?: string;
  hash_confidence?: string;
  library_name: string;
  library_type_name: string;

  // NEW: Reading direction fields for manga/comics
  manga_type?: "unknown" | "no" | "yes" | "yes_and_right_to_left";
  reading_direction?: "auto" | "ltr" | "rtl" | "vertical";

  // NEW: Universal metadata fields (apply to ebooks, audiobooks, comics)
  series_count?: number;
  volume?: number;
  imprint?: string;
  age_rating?: string; // 'Everyone' | 'Teen' | 'Mature' | 'Adult'
  web_url?: string;
  metadata_notes?: string;
  community_rating?: number;

  // NEW: Comic-specific fields
  story_arc?: string;
  is_black_and_white?: boolean;
  alternate_info?: {
    alternate_series?: string;
    alternate_number?: number;
    alternate_count?: number;
  };
  scan_information?: string;
  summary?: string;
}

Add these fields at the end of the interface, just before the closing }.

5.2 Verify API Response Handling

Files to Check:

internal/handlers/media.go - Ensure SearchMediaItems() includes new columns
internal/handlers/collections.go - Ensure BookInfo can include reading direction if needed
internal/database/queries.sql - Ensure queries select manga_type and reading_direction

Note: SQL queries should automatically include new columns after sqlc generate is run.

Phase 6: Testing

6.1 Unit Tests

File: internal/services/media_scanner_comic_test.go

New Test Cases to Add:

func TestNormalizeMangaType(t *testing.T) {
    tests := []struct {
        name     string
        input    string
        expected string
    }{
        {"Unknown", "Unknown", "unknown"},
        {"No", "No", "no"},
        {"Yes", "Yes", "yes"},
        {"YesAndRightToLeft", "YesAndRightToLeft", "yes_and_right_to_left"},
        {"Lowercase yesandrighttoleft", "yesandrighttoleft", "yes_and_right_to_left"},
        {"With spaces", "Yes And Right To Left", "yes_and_right_to_left"},
        {"Invalid", "invalid", "unknown"},
        {"Empty", "", "unknown"},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            result := normalizeMangaType(tt.input)
            if result != tt.expected {
                t.Errorf("normalizeMangaType(%q) = %q; want %q", tt.input, result, tt.expected)
            }
        })
    }
}

func TestDetermineReadingDirection(t *testing.T) {
    tests := []struct {
        name     string
        manga    string
        language string
        tags     string
        genre    string
        expected string
    }{
        {"Explicit RTL", "YesAndRightToLeft", "en", "", "", "rtl"},
        {"Explicit LTR (Yes)", "Yes", "ja", "", "", "ltr"},
        {"Explicit LTR (No)", "No", "en", "", "", "ltr"},
        {"Japanese heuristic", "Unknown", "ja", "", "", "rtl"},
        {"Japanese with full code", "Unknown", "jpn", "", "", "rtl"},
        {"Webtoon Korean", "Unknown", "ko", "Webtoon", "", "vertical"},
        {"Manhwa in tags", "Unknown", "ko", "", "Manhwa", "vertical"},
        {"Manga + Japanese", "Unknown", "ja", "Manga", "", "rtl"},
        {"Western default", "Unknown", "en", "", "", "ltr"},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            comicInfo := &ComicInfo{
                Manga:       tt.manga,
                LanguageISO: tt.language,
                Tags:        tt.tags,
                Genre:       tt.genre,
            }
            result := determineReadingDirection(comicInfo)
            if result != tt.expected {
                t.Errorf("determineReadingDirection() = %q; want %q", result, tt.expected)
            }
        })
    }
}

func TestMergeMetadata(t *testing.T) {
    // Test case 1: metadata.opf exists, ComicInfo.xml exists
    // Expected: metadata.opf fields take priority, except reading_direction from ComicInfo

    // Test case 2: metadata.opf exists, ComicInfo.xml missing
    // Expected: only metadata.opf fields used

    // Test case 3: metadata.opf missing, ComicInfo.xml exists
    // Expected: ComicInfo.xml fields used

    // Test case 4: Both missing
    // Expected: fallback to filename
}

6.2 Integration Tests

File: cmd/server/tests/comic_metadata_test.go

Create New File: Following test_helpers pattern

Test Structure:

package tests

import (
	"context"
	"testing"

	"bookhoard/internal/database"

	"github.com/jackc/pgx/v5/pgtype"
	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
)

// TestComicMetadataExtraction tests that comic metadata fields are stored correctly
func TestComicMetadataExtraction(t *testing.T) {
	setup := setupDeviceTest(t)
	defer setup.Server.Close()

	ctx := context.Background()
	libraryID := setup.CreateLibrary(t, "Comic Test Library", "comic")

	t.Run("CBZ with RTL manga", func(t *testing.T) {
		// Insert test media item with full comic metadata
		_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
			LibraryID:        libraryID,
			Title:            "Test Manga",
			FilePath:         "/test/manga.cbz",
			MangaType:        pgtype.Text{String: "yes_and_right_to_left", Valid: true},
			ReadingDirection: pgtype.Text{String: "rtl", Valid: true},
			SeriesCount:      pgtype.Int4{Int32: 12, Valid: true},
			Volume:           pgtype.Int4{Int32: 1, Valid: true},
			StoryArc:         pgtype.Text{String: "The Dark Phoenix Saga", Valid: true},
			AgeRating:        pgtype.Text{String: "Teen", Valid: true},
			CommunityRating:  pgtype.Float8{Float64: 8.5, Valid: true},
			Imprint:          pgtype.Text{String: "Shonen Jump", Valid: true},
			IsBlackAndWhite:  pgtype.Bool{Bool: false, Valid: true},
		})
		require.NoError(t, err)

		// Query it back
		items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
			LibraryID: libraryID,
		})
		require.NoError(t, err)
		require.Greater(t, len(items), 0)

		item := items[0]
		assert.Equal(t, "Test Manga", item.Title)
		assert.Equal(t, "yes_and_right_to_left", item.MangaType.String)
		assert.Equal(t, "rtl", item.ReadingDirection.String)
		assert.Equal(t, int32(12), item.SeriesCount.Int32)
		assert.Equal(t, int32(1), item.Volume.Int32)
		assert.Equal(t, "The Dark Phoenix Saga", item.StoryArc.String)
		assert.Equal(t, "Teen", item.AgeRating.String)
		assert.Equal(t, 8.5, item.CommunityRating.Float64)
		assert.Equal(t, "Shonen Jump", item.Imprint.String)
		assert.False(t, item.IsBlackAndWhite.Bool)
	})

	t.Run("CBZ with Western comic", func(t *testing.T) {
		_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
			LibraryID:        libraryID,
			Title:            "Test Comic",
			FilePath:         "/test/comic.cbz",
			MangaType:        pgtype.Text{String: "no", Valid: true},
			ReadingDirection: pgtype.Text{String: "ltr", Valid: true},
			Imprint:          pgtype.Text{String: "Vertigo", Valid: true},
			IsBlackAndWhite:  pgtype.Bool{Bool: true, Valid: true},
			StoryArc:         pgtype.Text{String: "Batman: Year One", Valid: true},
			SeriesCount:      pgtype.Int4{Int32: 4, Valid: true},
		})
		require.NoError(t, err)

		items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
			LibraryID: libraryID,
		})
		require.NoError(t, err)

		item := items[0]
		assert.Equal(t, "no", item.MangaType.String)
		assert.Equal(t, "ltr", item.ReadingDirection.String)
		assert.Equal(t, "Vertigo", item.Imprint.String)
		assert.True(t, item.IsBlackAndWhite.Bool)
		assert.Equal(t, "Batman: Year One", item.StoryArc.String)
		assert.Equal(t, int32(4), item.SeriesCount.Int32)
	})

	t.Run("Comic with minimal metadata", func(t *testing.T) {
		_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
			LibraryID: libraryID,
			Title:     "Minimal Comic",
			FilePath:  "/test/minimal.cbz",
			// Only required fields - comic metadata should default appropriately
		})
		require.NoError(t, err)

		items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
			LibraryID: libraryID,
		})
		require.NoError(t, err)

		item := items[0]
		assert.Equal(t, "Minimal Comic", item.Title)
		// Verify defaults
		assert.Equal(t, "unknown", item.MangaType.String)
		assert.Equal(t, "auto", item.ReadingDirection.String)
	})
}

// TestReadingDirectionAPI tests reading direction in API responses
func TestReadingDirectionAPI(t *testing.T) {
	setup := setupDeviceTest(t)
	defer setup.Server.Close()

	ctx := context.Background()
	libraryID := setup.CreateLibrary(t, "Reading Direction Test Library", "comic")

	// Create test items with different reading directions
	testCases := []struct {
		title string
		manga string
		dir   string
	}{
		{"Japanese Manga", "yes_and_right_to_left", "rtl"},
		{"Western Comic", "no", "ltr"},
		{"Webtoon", "unknown", "auto"},
	}

	for _, tc := range testCases {
		_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
			LibraryID:        libraryID,
			Title:            tc.title,
			FilePath:         "/test/" + tc.title + ".cbz",
			MangaType:        pgtype.Text{String: tc.manga, Valid: true},
			ReadingDirection: pgtype.Text{String: tc.dir, Valid: true},
		})
		require.NoError(t, err)
	}

	t.Run("Search API includes reading_direction", func(t *testing.T) {
		items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
			LibraryID: libraryID,
		})
		require.NoError(t, err)
		require.Len(t, items, 3)

		// Verify all items have reading direction set
		for _, item := range items {
			assert.NotEmpty(t, item.ReadingDirection.String)
			assert.NotEmpty(t, item.MangaType.String)
			assert.True(t, item.ReadingDirection.Valid)
		}
	})

	t.Run("Filter by reading_direction - RTL only", func(t *testing.T) {
		// Query all items and filter in-memory
		items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
			LibraryID: libraryID,
		})
		require.NoError(t, err)

		// Count RTL items
		rtlCount := 0
		for _, item := range items {
			if item.ReadingDirection.String == "rtl" {
				rtlCount++
			}
		}
		assert.Equal(t, 1, rtlCount)
	})

	t.Run("Verify all reading directions present", func(t *testing.T) {
		items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
			LibraryID: libraryID,
		})
		require.NoError(t, err)

		directions := make(map[string]bool)
		for _, item := range items {
			directions[item.ReadingDirection.String] = true
		}

		assert.True(t, directions["rtl"])
		assert.True(t, directions["ltr"])
		assert.True(t, directions["auto"])
	})
}

// TestUniversalMetadataFields tests universal fields apply to all formats
func TestUniversalMetadataFields(t *testing.T) {
	setup := setupDeviceTest(t)
	defer setup.Server.Close()

	ctx := context.Background()

	// Test with both comic and ebook libraries
	comicLibraryID := setup.CreateLibrary(t, "Comic Library", "comic")
	ebookLibraryID := setup.CreateLibrary(t, "Ebook Library", "ebook")

	t.Run("Comic with universal fields", func(t *testing.T) {
		_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
			LibraryID:       comicLibraryID,
			Title:           "Comic with Universal Metadata",
			FilePath:        "/test/comic.cbz",
			SeriesCount:     pgtype.Int4{Int32: 10, Valid: true},
			Volume:          pgtype.Int4{Int32: 2, Valid: true},
			Imprint:         pgtype.Text{String: "DC Black Label", Valid: true},
			AgeRating:       pgtype.Text{String: "Mature", Valid: true},
			WebURL:          pgtype.Text{String: "https://example.com/comic", Valid: true},
			CommunityRating: pgtype.Float8{Float64: 9.2, Valid: true},
		})
		require.NoError(t, err)

		items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
			LibraryID: comicLibraryID,
		})
		require.NoError(t, err)

		item := items[0]
		assert.Equal(t, int32(10), item.SeriesCount.Int32)
		assert.Equal(t, int32(2), item.Volume.Int32)
		assert.Equal(t, "DC Black Label", item.Imprint.String)
		assert.Equal(t, "Mature", item.AgeRating.String)
		assert.Equal(t, "https://example.com/comic", item.WebUrl.String)
		assert.Equal(t, 9.2, item.CommunityRating.Float64)
	})

	t.Run("Ebook with universal fields", func(t *testing.T) {
		_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
			LibraryID:       ebookLibraryID,
			Title:           "Ebook with Universal Metadata",
			FilePath:        "/test/book.epub",
			SeriesCount:     pgtype.Int4{Int32: 7, Valid: true},
			Volume:          pgtype.Int4{Int32: 1, Valid: true},
			Imprint:         pgtype.Text{String: "HarperCollins", Valid: true},
			AgeRating:       pgtype.Text{String: "Everyone", Valid: true},
			WebURL:          pgtype.Text{String: "https://example.com/book", Valid: true},
			CommunityRating: pgtype.Float8{Float64: 4.5, Valid: true},
		})
		require.NoError(t, err)

		items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
			LibraryID: ebookLibraryID,
		})
		require.NoError(t, err)

		item := items[0]
		assert.Equal(t, int32(7), item.SeriesCount.Int32)
		assert.Equal(t, int32(1), item.Volume.Int32)
		assert.Equal(t, "HarperCollins", item.Imprint.String)
		assert.Equal(t, "Everyone", item.AgeRating.String)
		assert.Equal(t, "https://example.com/book", item.WebUrl.String)
		assert.Equal(t, 4.5, item.CommunityRating.Float64)
	})
}

// TestComicSpecificFields tests comic-specific fields
func TestComicSpecificFields(t *testing.T) {
	setup := setupDeviceTest(t)
	defer setup.Server.Close()

	ctx := context.Background()
	libraryID := setup.CreateLibrary(t, "Comic Library", "comic")

	t.Run("Alternate series info as JSONB", func(t *testing.T) {
		alternateInfo := `{"alternate_series":"Ultimate X-Men","alternate_number":1,"alternate_count":12}`

		_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
			LibraryID:      libraryID,
			Title:          "X-Men with Alternate Series",
			FilePath:       "/test/xmen.cbz",
			AlternateInfo:  []byte(alternateInfo),
			ScanInformation: pgtype.Text{String: "Scanned by Minutemen", Valid: true},
			Summary:          pgtype.Text{String: "Professor X creates mutant team", Valid: true},
			MetadataNotes:    pgtype.Text{String: "From collection", Valid: true},
		})
		require.NoError(t, err)

		items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
			LibraryID: libraryID,
		})
		require.NoError(t, err)

		item := items[0]
		assert.NotNil(t, item.AlternateInfo)
		assert.JSONEq(t, alternateInfo, string(item.AlternateInfo))
		assert.Equal(t, "Scanned by Minutemen", item.ScanInformation.String)
		assert.Equal(t, "Professor X creates mutant team", item.Summary.String)
		assert.Equal(t, "From collection", item.MetadataNotes.String)
	})
}

Important: Use setupDeviceTest(t) from cmd/server/tests/test_helpers_test.go

setupDeviceTest returns *TestDeviceSetup which has the CreateLibrary() helper method
Internally calls setupTestServer() to set up the database and server
Call setupDeviceTest(t) ONCE per test function, not per subtest
The device setup provides user, auth tokens, and library creation helpers (you don't need to use the device-specific features)
Verify CommunityRating is pgtype.Float8 (DOUBLE PRECISION) in all tests

Phase 7: Bruno API Tests

7.1 Create Bruno YAML Files

Directory: bruno/media-items/

Files to Create:

`bruno/media-items/search-with-reading-direction.yml`

meta:
  name: Search media items with reading direction filter
  type: http
  seq: 1
config:
  test:
    filter: search-reading-direction
    workflow: search

http:
  method: GET
  url: "{{baseUrl}}/api/media-items?library_id={{libraryId}}&reading_direction=rtl"
  headers:
    Authorization: "Bearer {{accessToken}}"

`bruno/media-items/create-comic-with-metadata.yml`

meta:
  name: Upload comic with ComicInfo.xml
  type: http
  seq: 2
config:
  test:
    filter: comic-upload
    workflow: upload

http:
  method: POST
  url: "{{baseUrl}}/api/media-items/upload"
  headers:
    Authorization: "Bearer {{accessToken}}"
    Content-Type: "multipart/form-data"
  body:
    form_data:
      library_id: "{{libraryId}}"
      file:
        type: file
        src: test-files/manga-rtl.cbz

Phase 8: Documentation

8.1 Update User Documentation

File: docs/user/manga-reading-direction.md

Create New File:

# Manga Reading Direction Support

Bookhoard automatically detects and stores the reading direction for manga and comics.

## Supported Reading Directions

- **LTR (Left-to-Right)**: Western comics, manhwa, some manga
- **RTL (Right-to-Left)**: Traditional Japanese manga
- **Vertical**: Webtoons, manhwa (Korean webcomics)
- **Auto**: System will determine direction automatically

## How Reading Direction is Detected

1. **ComicInfo.xml**: If your comic archive contains `ComicInfo.xml` with the `Manga` field set to `YesAndRightToLeft`, Bookhoard will mark it as RTL
2. **Language**: Japanese comics without metadata are assumed to be RTL
3. **Genre**: Comics tagged as "webtoon" or "manhwa" are marked as vertical
4. **Manual**: You can manually override the reading direction in the metadata editor

## ComicInfo.xml Support

Bookhoard supports the industry-standard `ComicInfo.xml` format for comic metadata. When importing comics:

- If `ComicInfo.xml` exists inside the archive, its metadata is extracted
- The `Manga` field is used to set reading direction
- If a Calibre `metadata.opf` file exists in the same folder, both files are merged:
  - Calibre metadata takes priority for title, author, publisher, etc.
  - ComicInfo.xml takes priority for reading direction (since Calibre doesn't support it)

## Metadata Editor

You can view and edit reading direction in the metadata editor:

1. Navigate to a manga or comic in your library
2. Click "Edit Metadata"
3. Change the "Reading Direction" field
4. Save to update the database (and optionally write back to the file)

## Device Sync

Reading direction is synced to your devices:

- **KOReader**: Reading direction is set in the document metadata
- **Kobo**: Reading direction is stored in the Kobo metadata
- **Web Reader**: The reader automatically adjusts page turn direction based on reading direction

## Troubleshooting

### My manga is showing the wrong reading direction

1. Check if the comic has `ComicInfo.xml` inside the archive
2. Open the archive and verify the `Manga` field value:
   - `YesAndRightToLeft` → RTL
   - `Yes` or `No` → LTR
3. If no metadata exists, manually edit the reading direction in Bookhoard

### How do I add ComicInfo.xml to my comics?

Use a comic metadata editor such as:

- **ComicTagger** (cross-platform)
- **ComicRack** (Windows)
- **Komga** (can generate metadata for your library)

### My Calibre library has metadata.opf files. Will they work?

Yes! Bookhoard intelligently merges Calibre `metadata.opf` and `ComicInfo.xml`:

- Calibre metadata is used for title, author, series, etc.
- ComicInfo.xml is used for reading direction
- You get the best of both worlds

8.2 Update API Documentation

File: docs/developer/api/media-items/search.md

Update: Add reading_direction parameter to query parameters section

Add:

### Query Parameters

| Parameter | Type | Description | Example |
|-----------|------|-------------|---------|
| library_id | string | Filter by library UUID | `?library_id=uuid` |
| reading_direction | string | Filter by reading direction: `auto`, `ltr`, `rtl`, `vertical` | `?reading_direction=rtl` |
| manga_type | string | Filter by raw manga type: `unknown`, `no`, `yes`, `yes_and_right_to_left` | `?manga_type=yes_and_right_to_left` |

File: docs/developer/api/media-items/object.md

Update: Add manga_type and reading_direction to MediaItemSummary object

Add:

### MediaItemSummary Object

| Field | Type | Description |
|-------|------|-------------|
| ... existing fields ... |
| manga_type | string | Raw `Manga` field from ComicInfo.xml: `unknown`, `no`, `yes`, `yes_and_right_to_left` |
| reading_direction | string | Computed reading direction: `auto`, `ltr` (left-to-right), `rtl` (right-to-left), `vertical` (webtoons) |

Phase 9: Build & Verification

9.1 Build Verification

Commands:

# Build backend
go build ./...

# Run tests
go test ./... -v

# Verify guidelines compliance
bash scripts/verify-guidelines.sh

9.2 Database Update

Option 1: Recreate Database (Recommended - Loses Data)

podman compose down -v  # Delete all volumes
podman compose up -d    # Start with fresh schema

Option 2: Manual Migration (Preserves Data)

podman exec bookhoard_db psql -U postgres -d bookhoard -c "
-- Add manga and comic metadata fields (14 new columns)
ALTER TABLE media_items
ADD COLUMN manga_type VARCHAR(30) DEFAULT 'unknown'
CHECK (manga_type IN ('unknown', 'no', 'yes', 'yes_and_right_to_left'));

ALTER TABLE media_items
ADD COLUMN reading_direction VARCHAR(20) DEFAULT 'auto'
CHECK (reading_direction IN ('auto', 'ltr', 'rtl', 'vertical'));

ALTER TABLE media_items
ADD COLUMN series_count INTEGER;

ALTER TABLE media_items
ADD COLUMN volume INTEGER;

ALTER TABLE media_items
ADD COLUMN imprint VARCHAR(255);

ALTER TABLE media_items
ADD COLUMN story_arc VARCHAR(255);

ALTER TABLE media_items
ADD COLUMN age_rating VARCHAR(20);

ALTER TABLE media_items
ADD COLUMN web_url VARCHAR(500);

ALTER TABLE media_items
ADD COLUMN metadata_notes TEXT;

ALTER TABLE media_items
ADD COLUMN is_black_and_white BOOLEAN DEFAULT FALSE;

ALTER TABLE media_items
ADD COLUMN community_rating DOUBLE PRECISION;

ALTER TABLE media_items
ADD COLUMN alternate_info JSONB;

ALTER TABLE media_items
ADD COLUMN scan_information TEXT;

ALTER TABLE media_items
ADD COLUMN summary TEXT;

-- Create indexes for efficient querying
CREATE INDEX IF NOT EXISTS idx_media_items_reading_direction
ON media_items(reading_direction)
WHERE reading_direction IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_media_items_story_arc
ON media_items(story_arc)
WHERE story_arc IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_media_items_imprint
ON media_items(imprint)
WHERE imprint IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_media_items_age_rating
ON media_items(age_rating)
WHERE age_rating IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_media_items_manga_type
ON media_items(manga_type)
WHERE manga_type IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_media_items_series_count
ON media_items(series_count)
WHERE series_count IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_media_items_volume
ON media_items(volume)
WHERE volume IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_media_items_alternate_info_gin
ON media_items USING GIN (alternate_info)
WHERE alternate_info IS NOT NULL;

-- Add column comments
COMMENT ON COLUMN media_items.manga_type IS 'Raw Manga field from ComicInfo.xml: unknown, no, yes, yes_and_right_to_left';
COMMENT ON COLUMN media_items.reading_direction IS 'Computed reading direction: auto, ltr, rtl, vertical';
COMMENT ON COLUMN media_items.series_count IS 'Total items in series (from ComicInfo.xml Count field or book series count)';
COMMENT ON COLUMN media_items.volume IS 'Volume/omnibus number for collected editions';
COMMENT ON COLUMN media_items.imprint IS 'Publisher imprint/subdivision (e.g., Vertigo, Icon, DC Black Label)';
COMMENT ON COLUMN media_items.story_arc IS 'Story arc name for grouping related issues (e.g., "The Dark Phoenix Saga", "Civil War")';
COMMENT ON COLUMN media_items.age_rating IS 'Age rating: Everyone, Teen, Mature, Adult';
COMMENT ON COLUMN media_items.web_url IS 'URL to info page (Goodreads, ComicVine, MangaUpdates, Audible, etc.)';
COMMENT ON COLUMN media_items.metadata_notes IS 'Notes from metadata files (ComicInfo.xml, EPUB, PDF) - distinct from user notes';
COMMENT ON COLUMN media_items.is_black_and_white IS 'Black and white flag (mostly comics)';
COMMENT ON COLUMN media_items.community_rating IS 'Community rating from metadata (scale 0.0-10.0) - distinct from user ratings';
COMMENT ON COLUMN media_items.alternate_info IS 'Alternate series information as JSONB: {alternate_series, alternate_number, alternate_count}';
COMMENT ON COLUMN media_items.scan_information IS 'Scan information (scanner group, resolution, etc.)';
COMMENT ON COLUMN media_items.summary IS 'Summary from ComicInfo.xml (may be merged with description from Calibre)';
"

Option 3: Using SQL File

# Save migration to file
cat > /tmp/add_comic_metadata.sql << 'EOF'
-- Add comprehensive metadata support (15 new columns)
-- Universal fields (apply to ebooks, audiobooks, comics)
ALTER TABLE media_items
ADD COLUMN manga_type VARCHAR(30) DEFAULT 'unknown'
CHECK (manga_type IN ('unknown', 'no', 'yes', 'yes_and_right_to_left'));

ALTER TABLE media_items
ADD COLUMN reading_direction VARCHAR(20) DEFAULT 'auto'
CHECK (reading_direction IN ('auto', 'ltr', 'rtl', 'vertical'));

ALTER TABLE media_items ADD COLUMN series_count INTEGER;
ALTER TABLE media_items ADD COLUMN volume INTEGER;
ALTER TABLE media_items ADD COLUMN imprint VARCHAR(255);
ALTER TABLE media_items ADD COLUMN age_rating VARCHAR(20);
ALTER TABLE media_items ADD COLUMN web_url VARCHAR(500);
ALTER TABLE media_items ADD COLUMN metadata_notes TEXT;
ALTER TABLE media_items ADD COLUMN community_rating DECIMAL(3,1);

-- Comic-specific fields
ALTER TABLE media_items ADD COLUMN story_arc VARCHAR(255);
ALTER TABLE media_items ADD COLUMN is_black_and_white BOOLEAN DEFAULT FALSE;
ALTER TABLE media_items ADD COLUMN alternate_info JSONB;
ALTER TABLE media_items ADD COLUMN scan_information TEXT;
ALTER TABLE media_items ADD COLUMN summary TEXT;

-- Create indexes
CREATE INDEX IF NOT EXISTS idx_media_items_reading_direction
ON media_items(reading_direction) WHERE reading_direction IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_media_items_story_arc
ON media_items(story_arc) WHERE story_arc IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_media_items_imprint
ON media_items(imprint) WHERE imprint IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_media_items_age_rating
ON media_items(age_rating) WHERE age_rating IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_media_items_manga_type
ON media_items(manga_type) WHERE manga_type IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_media_items_series_count
ON media_items(series_count) WHERE series_count IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_media_items_volume
ON media_items(volume) WHERE volume IS NOT NULL;

CREATE INDEX IF NOT EXISTS idx_media_items_alternate_info_gin
ON media_items USING GIN (alternate_info) WHERE alternate_info IS NOT NULL;

-- Add comments
COMMENT ON COLUMN media_items.manga_type IS 'Raw Manga field from ComicInfo.xml: unknown, no, yes, yes_and_right_to_left';
COMMENT ON COLUMN media_items.reading_direction IS 'Computed reading direction: auto, ltr, rtl, vertical';
COMMENT ON COLUMN media_items.series_count IS 'Total items in series';
COMMENT ON COLUMN media_items.volume IS 'Volume/omnibus number for collected editions';
COMMENT ON COLUMN media_items.imprint IS 'Publisher imprint (e.g., Vertigo, Icon)';
COMMENT ON COLUMN media_items.age_rating IS 'Age rating: Everyone, Teen, Mature, Adult';
COMMENT ON COLUMN media_items.web_url IS 'URL to info page (Goodreads, ComicVine, MangaUpdates, Audible, etc.)';
COMMENT ON COLUMN media_items.metadata_notes IS 'Notes from metadata files (not user notes)';
COMMENT ON COLUMN media_items.community_rating IS 'Community rating (0.0-10.0)';
COMMENT ON COLUMN media_items.story_arc IS 'Story arc name for grouping';
COMMENT ON COLUMN media_items.is_black_and_white IS 'Black and white flag (mostly comics)';
COMMENT ON COLUMN media_items.alternate_info IS 'Alternate series as JSONB';
COMMENT ON COLUMN media_items.scan_information IS 'Scan information';
COMMENT ON COLUMN media_items.summary IS 'Summary from ComicInfo.xml';
EOF

# Apply migration
podman exec -i bookhoard_db psql -U postgres -d bookhoard < /tmp/add_comic_metadata.sql

Phase 10: Git Commit Structure

Commit Strategy Overview

Principle: Make good organized git commits with detailed messages for the entire project (not just what you changed). Run git add, commit, push sequentially as separate commands. Do not make one giant commit unless you are sure all files changed are for the same related edit.

Total Commits: 10 separate commits (organized by logical grouping)

Commits (Sequential, No `&&` Chaining)

IMPORTANT: Run these commands SEQUENTIALLY, waiting for each to complete before starting the next.

# Commit 1: Database schema changes
git add database/schema/schema.sql
git commit -m "feat: add comprehensive comic metadata support to media_items table

Add 16 new columns to support all ComicInfo.xml v2.0 fields:
- manga_type: Raw Manga field (unknown, no, yes, yes_and_right_to_left)
- reading_direction: Computed direction (auto, ltr, rtl, vertical)
- series_count: Total issues in series
- volume: Volume number for collected editions
- imprint: Publisher imprint (e.g., Vertigo, Icon)
- story_arc: Story arc name for grouping issues
- age_rating: Age rating (Everyone, Teen, Mature, Adult)
- genre_comic: Comic-specific genre
- web_url: URL to comic info page
- metadata_notes: Notes from ComicInfo.xml (distinct from user notes)
- is_black_and_white: B/W flag
- community_rating: Community rating (0-10 scale)
- alternate_info: JSONB for alternate series data
- scan_information: Scan info (scanner, resolution)
- summary: Summary from ComicInfo.xml

Add 8 indexes for efficient comic and universal metadata queries:
- idx_media_items_reading_direction
- idx_media_items_story_arc
- idx_media_items_imprint
- idx_media_items_age_rating
- idx_media_items_manga_type
- idx_media_items_series_count (NEW)
- idx_media_items_volume (NEW)
- idx_media_items_alternate_info_gin

Supports all 19 ComicInfo.xml fields plus universal fields for all formats:
Comic-specific: reading direction, story arc, scan information, alternate info, B/W flag
Universal (ebooks, audiobooks, comics): age rating, series count, volume, imprint, web URL, metadata notes, community rating

Genre/tag processing: ALL genres appear in tags array without duplication
- Uses existing `genre` column for primary genre (first genre tag)
- Populates `tags` array with all genres from metadata files
- Works for EPUB, ComicInfo.xml, PDF metadata
- No genre_comic column needed (reuses existing genre column)"

git add internal/database/
git commit -m "chore: regenerate database models for comprehensive metadata support

- Regenerate models.go with sqlc after schema changes
- Update CreateMediaItemParams with 15 new fields (was 16, removed genre_comic)
- Update all queries that return MediaItems to include new columns
- Support pgtype types for TEXT, JSONB, DECIMAL, BOOLEAN fields"

git add internal/services/media_scanner.go
git commit -m "feat: add smart metadata merging with genre/tag processing

- Add all 19 fields to ComicInfo struct (complete v2.0 support)
- Add universal + comic fields to MediaMetadata struct (15 fields total)
- Implement comprehensive mergeMetadata() for intelligent merging:
  - Priority: Calibre metadata.opf → ComicInfo.xml → folder structure → filename
  - Always extract comic fields even when metadata.opf exists
  - Merge Characters, Teams, Locations into tags array via extractGenreTagsFromComicInfo()
  - Build alternate_info JSONB from AlternateSeries/Number/Count
- Add genre/tag processing functions (universal for ALL formats):
  - processGenresAndTags(): Ensures ALL genres appear in tags without duplication
  - extractGenreTagsFromEPUB(): Extract genres from EPUB <dc:subject> tags
  - extractGenreTagsFromComicInfo(): Extract Genre + Tags + Characters + Teams + Locations
  - Uses existing `genre` column for primary genre (first genre tag)
  - Populates `tags` array with all genres (no duplicates via containsTag())
- Add helper functions:
  - normalizeMangaType(): Standardize Manga field values
  - determineReadingDirection(): Compute from Manga + language + genre
  - normalizeAgeRating(): Standardize age rating values
  - containsTag(): Prevent duplicate tags
- Update extractMetadata() to use smart merging
- Remove duplicate comic metadata extraction in processMediaFile()
- Calibre metadata.opf takes priority for basic fields
- ComicInfo.xml fills gaps and provides comic-specific fields
- Ensures all 19 ComicInfo.xml fields + 5 universal fields are extracted

Supports complete ComicInfo.xml v2.0 field extraction:
- Series organization (Count, Volume, AlternateSeries)
- Publisher hierarchy (Publisher, Imprint)
- Content classification (Genre, Manga, AgeRating, StoryArc, SeriesGroup)
- Narrative elements (Summary, Notes, Characters, Teams, Locations)
- Publication metadata (Year, Month, Day, Web, PageCount, LanguageISO)
- Credits (Writer, Penciller, Inker, Colorist, Letterer, CoverArtist, Editor)
- Format (BlackAndWhite, Manga)
- Community data (CommunityRating, MainCharacterOrTeam, Review)
- Technical (ScanInformation)

Universal metadata fields (apply to ebooks, audiobooks, comics):
- age_rating: Age-based content classification
- series_count: Total items in series
- volume: Collected edition/omnibus number
- imprint: Publisher subdivision
- web_url: Info page URL (Goodreads, Audible, ComicVine, etc.)
- metadata_notes: Notes from metadata files (not user notes)
- community_rating: Pre-existing ratings from metadata"

git add internal/services/media_scanner_comic_test.go
git commit -m "test: add unit tests for metadata extraction and genre/tag processing

- Add TestNormalizeMangaType with all enum values and edge cases
- Add TestDetermineReadingDirection with Manga, language, genre heuristics
- Add TestNormalizeAgeRating with all rating values
- Add TestProcessGenresAndTags for universal genre/tag logic
- Add TestExtractGenreTagsFromEPUB for EPUB subject processing
- Add TestExtractGenreTagsFromComicInfo for comic tag processing
- Add TestMergeMetadata for metadata.opf + ComicInfo.xml merge scenarios
- Test all 19 ComicInfo.xml fields + 5 universal fields are extracted correctly
- Test alternate_info JSONB serialization
- Test tag deduplication (no duplicate genres in tags array)
- Test genre → primary genre, all genres → tags logic
- Cover RTL, LTR, vertical reading direction detection
- Test webtoon, manhwa, Japanese manga detection
- Test age rating normalization (Everyone, Teen, Mature, Adult)
- Test community rating extraction
- Test series count and volume extraction"

git add cmd/server/tests/comic_metadata_test.go
git commit -m "test: add integration tests for comic metadata extraction

- Add TestComicMetadataExtraction for CBZ with/without ComicInfo.xml
- Test all 19 fields are stored correctly in database
- Add TestCalibreComicMerge for metadata.opf + ComicInfo.xml merging
- Verify priority: metadata.opf → ComicInfo.xml → folder structure
- Add TestReadingDirectionAPI for API response validation
- Test alternate_info JSONB in API responses
- Test search by story_arc, imprint, age_rating
- Test series_count and volume in API responses
- Test Characters/Teams/Locations merged into tags
- Use setupTestServer() helper from test_helpers
- Test no user, user, and admin contexts"

git add web/src/types/api.d.ts
git commit -m "feat: add all comic metadata fields to MediaItemSummary type

- Add manga_type field with enum values
- Add reading_direction field with ltr/rtl/vertical/auto options
- Add series_count, volume, imprint, story_arc
- Add age_rating, genre_comic, web_url
- Add metadata_notes (distinct from user notes)
- Add is_black_and_white, community_rating
- Add alternate_info JSONB object
- Add scan_information, summary
- Ensure TypeScript types match Go database models
- Support all 15 new metadata fields (5 universal + 10 comic-specific)"

git add bruno/media-items/
git commit -m "test: add Bruno API tests for comic metadata feature

- Add search-with-reading-direction.yml for filtering by direction
- Add search-by-story-arc.yml for story arc queries
- Add search-by-imprint.yml for imprint filtering
- Add search-by-age-rating.yml for age rating queries (universal)
- Add search-by-series-count.yml for series filtering (universal)
- Add create-comic-with-metadata.yml for uploading with ComicInfo.xml
- Test API contracts for all new fields (universal + comic-specific)
- Verify JSONB serialization of alternate_info
- Test genre/tag processing (all genres appear in tags without duplication)"

git add docs/user/manga-reading-direction.md
git commit -m "docs: add comprehensive user documentation for metadata features

- Explain all 19 ComicInfo.xml fields
- Document reading direction detection (Manga + language + genre)
- Document smart merging with Calibre metadata.opf
- Explain universal fields (age_rating, series_count, volume, imprint, web_url)
- Explain comic-specific fields (story arc, scan_information, alternate_info)
- Document genre/tag processing (all genres appear in tags)
- Add troubleshooting guide for missing metadata
- Document metadata editor usage
- Explain how to add ComicInfo.xml to comics
- List recommended metadata editor tools
- Document device sync for reading direction"

git add docs/developer/api/media-items/
git commit -m "docs: update API documentation for all metadata fields

- Add reading_direction, manga_type query parameters
- Add story_arc, imprint, age_rating filters
- Add series_count, volume, web_url filters
- Update MediaItemSummary object with all 15 new fields
- Document alternate_info JSONB structure
- Document genre/tag processing behavior
- Document enum values for manga_type and reading_direction
- Document age_rating enum values
- Add examples for filtering by metadata
- Note universal fields apply to ebooks, audiobooks, comics"

git push

Summary of Changes

Files Modified

database/schema/schema.sql - Add 15 new columns, 8 indexes, column comments
internal/database/models.go - Regenerated by sqlc with new fields
internal/services/media_scanner.go - Smart merging logic + genre/tag processing
internal/services/media_scanner_comic_test.go - Unit tests for all new fields
web/src/types/api.d.ts - TypeScript types with all metadata
docs/user/manga-reading-direction.md - User docs (new file)
docs/developer/api/media-items/*.md - API docs updates

Files Created

cmd/server/tests/comic_metadata_test.go - Integration tests for all fields
bruno/media-items/search-with-reading-direction.yml - API test
bruno/media-items/create-comic-with-metadata.yml - API test
bruno/media-items/search-by-story-arc.yml - API test (new)
bruno/media-items/search-by-imprint.yml - API test (new)
bruno/media-items/search-by-age-rating.yml - API test (new)
bruno/media-items/search-by-series-count.yml - API test (new)

Key Features

✅ Complete ComicInfo.xml Support: Extracts ALL 19 fields from ComicInfo.xml v2.0 ✅ Universal Metadata Fields: 5 fields apply to ALL formats (ebooks, audiobooks, comics):

age_rating: Age-based content classification
series_count: Total items in series
volume: Collected edition/omnibus number
imprint: Publisher subdivision
web_url: Info page URL (Goodreads, Audible, ComicVine, etc.) ✅ Comic-Specific Fields: 10 fields for comics:
Reading direction (manga_type, reading_direction)
Story arc (story_arc)
Scan info (scan_information)
Alternate series (alternate_info JSONB)
Format (is_black_and_white)
Metadata (summary, metadata_notes, community_rating) ✅ Smart Metadata Merging: Calibre metadata.opf + ComicInfo.xml + folder structure ✅ Reading Direction Detection: From Manga field + language heuristics + genre tags ✅ Genre/Tag Processing: ALL genres appear in tags array without duplication
Uses existing genre column for primary genre (first genre tag)
Processes EPUB <dc:subject> tags
Processes ComicInfo Genre + Tags + Characters + Teams + Locations
Deduplication via containsTag() helper ✅ Database Storage: Raw manga_type + computed reading_direction + 13 other fields ✅ Full Stack: Database → Go → API → TypeScript → Frontend ✅ Testing: Unit tests, integration tests, Bruno API tests ✅ Documentation: User guide, API reference, troubleshooting

All 19 ComicInfo.xml Fields Extracted

Field	Database Column	Type	Description
Title	title	VARCHAR(255)	Already exists
Series	series	VARCHAR(255)	Already exists
Number	series_number	INTEGER	Already exists
Count	series_count	INTEGER	NEW: Total issues in series (UNIVERSAL)
Volume	volume	INTEGER	NEW: Volume number (UNIVERSAL)
AlternateSeries	alternate_info	JSONB	NEW: Alternate series info (comic-specific)
AlternateNumber	alternate_info	JSONB	NEW: Alternate number (comic-specific)
AlternateCount	alternate_info	JSONB	NEW: Alternate count (comic-specific)
Summary	summary	TEXT	NEW: Comic summary (can merge with description)
Notes	metadata_notes	TEXT	NEW: Notes from metadata files (UNIVERSAL, distinct from user notes)
Year	copyright_year	INTEGER	Already exists
Month	(derived)	-	Stored in date_published
Day	(derived)	-	Stored in date_published
Writer	author	VARCHAR(255)	Already exists
Penciller	contributors	TEXT[]	Merged into existing
Inker	contributors	TEXT[]	Merged into existing
Colorist	contributors	TEXT[]	Merged into existing
Letterer	contributors	TEXT[]	Merged into existing
CoverArtist	contributors	TEXT[]	Merged into existing
Publisher	publisher	VARCHAR(255)	Already exists
Imprint	imprint	VARCHAR(255)	NEW: Publisher imprint (UNIVERSAL)
Genre	genre	VARCHAR(100)	Already exists + enhanced via processGenresAndTags()
Tags	tags	TEXT[]	Already exists + enhanced with Characters/Teams/Locations
Web	web_url	VARCHAR(500)	NEW: Info page URL (UNIVERSAL)
PageCount	page_count	INTEGER	Already exists
LanguageISO	language	VARCHAR(10)	Already exists
Manga	manga_type	VARCHAR(30)	NEW: Raw manga field (comic-specific)
Derived	reading_direction	VARCHAR(20)	NEW: Computed from Manga field (comic-specific)
BlackAndWhite	is_black_and_white	BOOLEAN	NEW: B/W flag (mostly comic-specific)
StoryArc	story_arc	VARCHAR(255)	NEW: Story arc name (comic-specific)
SeriesGroup	(future)	-	Not stored (can add later)
AgeRating	age_rating	VARCHAR(20)	NEW: Age rating (UNIVERSAL)
CommunityRating	community_rating	DOUBLE PRECISION	NEW: 0-10 rating (UNIVERSAL, from metadata files)
MainCharacterOrTeam	tags	TEXT[]	Merged into tags via processGenresAndTags()
Review	(future)	-	Not stored (user notes exist)
ScanInformation	scan_information	TEXT	NEW: Scan info (comic-specific)
Characters	tags	TEXT[]	Merged into tags via processGenresAndTags()
Teams	tags	TEXT[]	Merged into tags via processGenresAndTags()
Locations	tags	TEXT[]	Merged into tags via processGenresAndTags()

Universal Fields (Apply to Ebooks, Audiobooks, Comics)

Field	Database Column	Formats	Examples
age_rating	age_rating	All	Everyone, Teen, Mature, Adult
series_count	series_count	All	"7" (Harry Potter), "12" (One Piece)
volume	volume	All	"1" (Omnibus), "2" (Collected Edition)
imprint	imprint	All	"Vertigo" (comics), "HarperCollinsEpic" (books)
web_url	web_url	All	Goodreads, Audible, ComicVine, MangaUpdates URLs
metadata_notes	metadata_notes	All	Publisher notes, scan info, EPUB annotations
community_rating	community_rating	All	Pre-existing ratings from metadata sources
Notes	metadata_notes	TEXT	NEW: Metadata notes (not user notes)
Year	copyright_year	INTEGER	Already exists
Month	(derived)	-	Stored in date_published
Day	(derived)	-	Stored in date_published
Writer	author	VARCHAR(255)	Already exists
Penciller	contributors	TEXT[]	Merged into existing
Inker	contributors	TEXT[]	Merged into existing
Colorist	contributors	TEXT[]	Merged into existing
Letterer	contributors	TEXT[]	Merged into existing
CoverArtist	contributors	TEXT[]	Merged into existing
Publisher	publisher	VARCHAR(255)	Already exists
Imprint	imprint	VARCHAR(255)	NEW: Publisher imprint (UNIVERSAL)
Genre	genre	VARCHAR(100)	Already exists + enhanced via processGenresAndTags()
Tags	tags	TEXT[]	Already exists + enhanced with Characters/Teams/Locations via processGenresAndTags()
Web	web_url	VARCHAR(500)	NEW: Info page URL (UNIVERSAL)
PageCount	page_count	INTEGER	Already exists
LanguageISO	language	VARCHAR(10)	Already exists
Manga	manga_type	VARCHAR(30)	NEW: Raw manga field
Derived	reading_direction	VARCHAR(20)	NEW: Computed from Manga field
BlackAndWhite	is_black_and_white	BOOLEAN	NEW: B/W flag
StoryArc	story_arc	VARCHAR(255)	NEW: Story arc name
SeriesGroup	(future)	-	Not stored (can add later)
AgeRating	age_rating	VARCHAR(20)	NEW: Age rating
CommunityRating	community_rating	DOUBLE PRECISION	NEW: 0-10 rating
MainCharacterOrTeam	tags	TEXT[]	Merged into tags
Review	(future)	-	Not stored (user notes exist)
ScanInformation	scan_information	TEXT	NEW: Scan info
Characters	tags	TEXT[]	Merged into tags
Teams	tags	TEXT[]	Merged into tags
Locations	tags	TEXT[]	Merged into tags

Testing Checklist

Before considering this feature complete:

Schema & Database

Database schema updated with 14 new columns (community_rating is DOUBLE PRECISION, not DECIMAL)
All 8 indexes created successfully (already in schema)
Column comments added for all new fields
Database models regenerated with sqlc after schema change (CommunityRating becomes pgtype.Float8)
Local database updated (Option 1: recreate database, or Option 2: manual migration)

Code Changes

ComicInfo struct updated with all 19 fields
MediaMetadata struct updated with 15 fields (5 universal + 10 comic-specific)
mergeMetadata() function implements smart merging
Genre/tag processing functions implemented:
- processGenresAndTags() for universal genre/tag logic
- extractGenreTagsFromEPUB() for EPUB subject tags
- extractGenreTagsFromComicInfo() for comic tags
normalizeMangaType() function handles all enum values
determineReadingDirection() implements heuristics
normalizeAgeRating() function standardizes ratings
containsTag() helper prevents duplicate tags
processMediaFile() updated with all new fields
TypeScript types updated with all metadata fields

Testing

Unit tests pass (go test ./internal/services/...)
Integration tests pass (go test ./cmd/server/tests/...)
Bruno API tests pass
Test coverage for all 19 ComicInfo.xml fields

Manual Testing - Core Functionality

Upload CBZ with ComicInfo.xml (Manga=YesAndRightToLeft)
Verify RTL reading direction in database
Upload CBZ with metadata.opf + ComicInfo.xml
Verify smart merging: Calibre fields + ComicInfo.xml fields
API responses include all new fields
Search by reading_direction filter works
Search by story_arc filter works
Search by imprint filter works
Search by age_rating filter works
Search by series_count (e.g., "show complete series")
Characters/Teams/Locations added to tags array

Manual Testing - Specific Fields (Universal + Comic)

series_count displays correctly in UI
volume displays correctly in UI
imprint displays and filters correctly (test with comics AND ebooks)
story_arc groups related comics
age_rating filters work (test with comics AND ebooks - parental controls)
web_url links work (test with Goodreads, ComicVine, etc.)
metadata_notes display (distinct from user notes)
community_rating displays (distinct from user ratings in media_ratings table)
is_black_and_white flag works
alternate_info JSONB stores/retrieves correctly
scan_information displays
summary merges with description appropriately

Manual Testing - Genre/Tag Processing

EPUB subjects: first subject → genre, all subjects → tags
ComicInfo Genre: → genre column
ComicInfo Tags: → tags array
ComicInfo Characters: → tags array
ComicInfo Teams: → tags array
ComicInfo Locations: → tags array
No duplicate genres in tags (deduplication works)
Primary genre set correctly (first genre tag wins)

Documentation

User documentation renders at /docs endpoint
API documentation updated with all new fields
Docs search finds new content
Code examples in docs work

Build & Verification

go build ./... succeeds
bash scripts/verify-guidelines.sh passes (0 errors)
No critical functionality broken
Git diff shows only intended changes

Document Version: 3.0 Created: March 29, 2026 Updated: March 29, 2026 Changes:

v2.0: Added all 19 ComicInfo.xml fields (from 2 to 16 columns)
v3.0: Made 5 fields universal (age_rating, series_count, volume, imprint, web_url)
v3.0: Removed genre_comic column (use existing genre + processGenresAndTags)
v3.0: Added genre/tag processing logic for ALL formats
v3.0: Total: 15 new columns (5 universal + 10 comic-specific) Status: Ready for Implementation

98 KiB Raw Blame History

Implementation Plan: Smart Metadata Merge & Complete ComicInfo.xml Support

⚠️ PLAN UPDATES (March 29, 2026)

Overview

Project Context

Phase 1: Database Schema Changes

1.1 Add Columns to media_items Table

1.2 Add Indexes for Comic and Universal Metadata Queries

1.3 Add Column Comments

1.4 Regenerate Database Models

Phase 2: Update Data Structures

2.1 Enhance ComicInfo Struct

2.2 Enhance MediaMetadata Struct

Phase 3: Smart Metadata Merging Logic

3.1 Create Metadata Merge Function

3.2 Create Reading Direction Helper Functions

3.3 Genre and Tag Processing Logic (Universal for ALL Formats)

3.4 Update extractMetadata() Function

Phase 4: Update Media Item Creation

4.1 Update CreateMediaItem SQL Query

4.2 Regenerate sqlc Models

4.3 Update CreateMediaItem Call

4.4 Remove Duplicate Comic Metadata Extraction

Phase 5: API Layer Updates

5.1 Update TypeScript API Types

5.2 Verify API Response Handling

Phase 6: Testing

6.1 Unit Tests

6.2 Integration Tests

Phase 7: Bruno API Tests

7.1 Create Bruno YAML Files

bruno/media-items/search-with-reading-direction.yml

bruno/media-items/create-comic-with-metadata.yml

Phase 8: Documentation

8.1 Update User Documentation

8.2 Update API Documentation

Phase 9: Build & Verification

9.1 Build Verification

9.2 Database Update

Phase 10: Git Commit Structure

Commit Strategy Overview

Commits (Sequential, No && Chaining)

Summary of Changes

Files Modified

Files Created

Key Features

All 19 ComicInfo.xml Fields Extracted

Universal Fields (Apply to Ebooks, Audiobooks, Comics)

Testing Checklist

Schema & Database

Code Changes

Testing

Manual Testing - Core Functionality

Manual Testing - Specific Fields (Universal + Comic)

Manual Testing - Genre/Tag Processing

Documentation

Build & Verification

98 KiB

Raw Blame History

1.1 Add Columns to `media_items` Table

2.1 Enhance `ComicInfo` Struct

2.2 Enhance `MediaMetadata` Struct

3.4 Update `extractMetadata()` Function

`bruno/media-items/search-with-reading-direction.yml`

`bruno/media-items/create-comic-with-metadata.yml`

Commits (Sequential, No `&&` Chaining)