Files
bookhoard/IMPLEMENTATION_PLAN_MERGE_METADATA_READING_DIRECTION.md
T
john-okeefe 008706a7fa docs: update implementation plan with DOUBLE PRECISION and complete integration tests
Phase 6.2 implementation: Add comprehensive integration test code to plan.

Documentation Updates:
- Changed community_rating from DECIMAL(3,1) to DOUBLE PRECISION throughout plan
- Fixed function name references: processNewMediaItems → processMediaFile (correct name)
- Added complete integration test implementation (TestComicMetadataExtraction,
  TestReadingDirectionAPI, TestUniversalMetadataFields, TestComicSpecificFields)
- Replaced skeleton TestMergeMetadata with actual test code using setupTestServer
- Added context-based location markers (3 lines before/after) for easier code navigation
- Removed TODO comment reference (doesn't exist in current code)
- Updated all line number references and added plan update summary
- Verified test helper usage: setupDeviceTest for library creation

Integration Tests Added:
- Full comic metadata field testing (manga_type, reading_direction, series_count, volume,
  imprint, age_rating, community_rating as pgtype.Float8, story_arc, is_black_and_white)
- Reading direction API testing (rtl, ltr, auto filtering)
- Universal field testing for both comic and ebook libraries
- Comic-specific field testing (alternate_info JSONB, scan_information, summary, metadata_notes)

All tests use proper test_helpers pattern with setupDeviceTest and verify DOUBLE PRECISION
storage for CommunityRating field.

Relates to: Phase 6.2 integration testing documentation
2026-03-29 21:12:27 -04:00

2427 lines
98 KiB
Markdown

<!-- markdownlint-disable MD013 -->
# Implementation Plan: Smart Metadata Merge & Complete ComicInfo.xml Support
**Scope:** Extract ALL 19 ComicInfo.xml v2.0 fields (not just reading direction)
**Status:** Updated - Full ComicInfo.xml support (March 29, 2026)
---
## ⚠️ PLAN UPDATES (March 29, 2026)
This implementation plan has been corrected with the following changes:
1. **Context-based location markers**: All line number references have been replaced with 3-line context markers (before/after) to make locations easier to find even after code insertions shift line numbers.
2. **Phase 1 (Database Schema)**: Marked as mostly complete. The schema columns, indexes, and comments are already in place. Only the SQL INSERT query needs updating.
3. **Phase 4.1 (SQL Query)**: Added the complete SQL query code showing the 14 new columns and parameters.
4. **Phase 4.4 (TODO Comment)**: Removed - this TODO comment doesn't exist in the current codebase.
5. **Phase 2.2 (MediaMetadata)**: Updated with exact location of the struct (starts around line 42, not line 101).
6. **Phase 3.4 (extractMetadata)**: Updated with context-based location and clarified that the fallback switch statement (`.epub`, `.pdf`) is removed as this logic moves into `mergeMetadata()`.
7. **Phase 4.4 (Duplicate Code)**: Renumbered from 4.5 and added context-based location markers.
8. **Fixed function name**: Corrected `processNewMediaItems()``processMediaFile()` throughout the plan. This is the actual function name in media_scanner.go.
9. **Changed community_rating to DOUBLE PRECISION**: Changed from `DECIMAL(3,1)` to `DOUBLE PRECISION` for simpler code. This eliminates the awkward `pgtype.Numeric` conversion and uses simple `pgtype.Float8` instead. Matches ComicInfo.xml float64 type naturally with no string conversion needed. The floating-point precision error is negligible for 0-10 ratings (< 0.00001%).
**What's Already Done:**
- Database schema columns (lines 137-161 in schema.sql)
- Database indexes (lines 424-461 in schema.sql)
- Go models regenerated (MediaItems struct lines 229-256 in models.go)
- CreateMediaItemParams struct (lines 586-599 in queries.sql.go)
**What Still Needs Doing:**
- Update CreateMediaItem SQL query in queries.sql
- Update ComicInfo struct with new fields
- Update MediaMetadata struct with new fields
- Create mergeMetadata() function
- Update extractMetadata() function
- Update CreateMediaItem() call with new parameters
- Remove duplicate comic extraction code
- Update TypeScript types
- Write tests
- Write documentation
---
## Overview
Add intelligent metadata merging for ALL media types (ebooks, comics, etc.) with comprehensive support for ALL ComicInfo.xml metadata fields (19 total fields). This feature will:
1. Parse `ComicInfo.xml` from comic archives (.cbz, .cbr, .cb7, .cbt)
2. Extract ALL metadata fields from ComicInfo.xml (reading direction, series info, imprint, story arc, age rating, etc.)
3. Implement smart metadata merging: Calibre `metadata.opf` + embedded metadata
4. Store both raw `manga_type` and computed `reading_direction` in database
5. Store universal fields that apply to ALL formats: age_rating, series_count, volume, imprint, web_url
6. Store comic-specific fields: story_arc, scan_information, alternate_info (JSONB), metadata_notes, community_rating
7. Implement genre/tag processing: ALL genres appear in tags array without duplication
8. Propagate all metadata through all layers: database → API → frontend
## Project Context
**Current State:**
- ✅ Parses Calibre `metadata.opf` sidecar files (priority)
- ✅ Parses `ComicInfo.xml` from comic archives (fallback only)
- ✅ Extracts basic comic metadata (title, series, number, publisher, writer)
-**Missing:** 13 additional ComicInfo.xml fields not extracted (series_count, volume, imprint, story_arc, age_rating, etc.)
-**Missing:** Universal fields that apply to all formats (age_rating, series_count, web_url, imprint, volume)
-**Missing:** `Manga` field in `ComicInfo` struct
-**Missing:** Reading direction in database schema
-**Missing:** Comic-specific fields (volume, series count, imprint, story arc, age rating, etc.)
-**Missing:** Smart metadata merging (currently: if `metadata.opf` exists, skip `ComicInfo.xml`)
-**Missing:** Reading direction in API responses
-**Missing:** Reading direction in frontend types
## Phase 1: Database Schema Changes
### 1.1 Add Columns to `media_items` Table
**File:** `database/schema/schema.sql`
**Location:** After line 133 (after `kobo_metadata JSONB`)
**Status:****Already completed** - All columns, indexes, and comments are in place
**Note:** The schema has already been updated with all 14 new columns. The only change needed is to ensure `community_rating` is `DOUBLE PRECISION` (which it already is in the current schema).
**Existing Schema (already in place):**
```sql
-- Manga and comic reading direction support
-- Stores raw Manga field from ComicInfo.xml
manga_type VARCHAR(30) DEFAULT 'unknown'
CHECK (manga_type IN ('unknown', 'no', 'yes', 'yes_and_right_to_left')),
-- Stores computed reading direction for queries/UI
reading_direction VARCHAR(20) DEFAULT 'auto'
CHECK (reading_direction IN ('auto', 'ltr', 'rtl', 'vertical')),
-- Universal series information (applies to ALL formats: ebooks, audiobooks, comics)
series_count INTEGER, -- Total items in series (from ComicInfo Count field, or book series count)
volume INTEGER, -- Volume/omnibus number for collected editions
-- Universal publisher and classification (applies to ALL formats)
imprint VARCHAR(255), -- Publisher imprint (e.g., Vertigo, HarperCollinsEpic)
age_rating VARCHAR(20), -- Age rating: Everyone, Teen, Mature, Adult (applies to all formats)
web_url VARCHAR(500), -- URL to info page (Goodreads, ComicVine, MangaUpdates, Audible, etc.)
-- Comic-specific fields
story_arc VARCHAR(255), -- Story arc name (e.g., "The Dark Phoenix Saga", "Civil War")
is_black_and_white BOOLEAN, -- Black and white flag (mostly comics, some illustrated books)
-- Additional metadata (applies to all formats)
metadata_notes TEXT, -- Notes from metadata files (ComicInfo.xml, EPUB, PDF) - distinct from user notes
community_rating DOUBLE PRECISION, -- Pre-existing community rating from metadata (0.0-10.0) - distinct from user ratings
-- Alternate series information (JSONB for flexible schema - comic-specific)
alternate_info JSONB, -- Stores AlternateSeries, AlternateNumber, AlternateCount
-- Example: {"alternate_series": "Ultimate X-Men", "alternate_number": 1, "alternate_count": 12}
-- Scan and publication metadata (comic-specific)
scan_information TEXT, -- Scan information (scanner group, resolution, etc.)
-- Summary (distinct from description - may merge with Calibre description)
summary TEXT, -- Summary from ComicInfo.xml (may be merged with description from Calibre)
```
**Summary of Phase 1:**
- ✅ All 14 columns already added to schema
- ✅ All 8 indexes already created
- ✅ All column comments already added
-`community_rating` is `DOUBLE PRECISION` (not DECIMAL)
- ⚠️ **Action needed:** Only if you haven't regenerated sqlc models yet
### 1.2 Add Indexes for Comic and Universal Metadata Queries
**Note:** The indexes for these fields have already been created in the schema. This section is for reference only.
**File:** `database/schema/schema.sql`
**Location:** Find the section with existing comic metadata indexes (already implemented)
**Existing Indexes (Already in schema.sql):**
```sql
-- Index for filtering by reading direction (for manga/comic libraries)
CREATE INDEX IF NOT EXISTS idx_media_items_reading_direction
ON media_items(reading_direction)
WHERE reading_direction IS NOT NULL;
-- Index for filtering by story arc (comic-specific)
CREATE INDEX IF NOT EXISTS idx_media_items_story_arc
ON media_items(story_arc)
WHERE story_arc IS NOT NULL;
-- Index for filtering by imprint (universal - all formats)
CREATE INDEX IF NOT EXISTS idx_media_items_imprint
ON media_items(imprint)
WHERE imprint IS NOT NULL;
-- Index for filtering by age rating (universal - all formats)
CREATE INDEX IF NOT EXISTS idx_media_items_age_rating
ON media_items(age_rating)
WHERE age_rating IS NOT NULL;
-- Index for filtering by manga type (comic-specific)
CREATE INDEX IF NOT EXISTS idx_media_items_manga_type
ON media_items(manga_type)
WHERE manga_type IS NOT NULL;
-- Index for filtering by series count (universal - all formats)
CREATE INDEX IF NOT EXISTS idx_media_items_series_count
ON media_items(series_count)
WHERE series_count IS NOT NULL;
-- Index for filtering by volume (universal - all formats)
CREATE INDEX IF NOT EXISTS idx_media_items_volume
ON media_items(volume)
WHERE volume IS NOT NULL;
-- GIN index for alternate_info JSONB queries (comic-specific)
CREATE INDEX IF NOT EXISTS idx_media_items_alternate_info_gin
ON media_items USING GIN (alternate_info)
WHERE alternate_info IS NOT NULL;
```
**Note:** These indexes are already in the schema and do not need to be added again.
### 1.3 Add Column Comments
**File:** `database/schema/schema.sql`
**Location:** After line 478 (after existing column comments)
**Changes:**
```sql
-- Comic-specific fields
COMMENT ON COLUMN media_items.manga_type IS 'Raw Manga field from ComicInfo.xml: unknown, no, yes, yes_and_right_to_left';
COMMENT ON COLUMN media_items.reading_direction IS 'Computed reading direction: auto, ltr (left-to-right), rtl (right-to-left), vertical (webtoons/manhwa)';
COMMENT ON COLUMN media_items.story_arc IS 'Story arc name for grouping related issues (e.g., "The Dark Phoenix Saga", "Civil War")';
COMMENT ON COLUMN media_items.is_black_and_white IS 'Black and white comic flag from ComicInfo.xml';
COMMENT ON COLUMN media_items.alternate_info IS 'Alternate series information as JSONB: {alternate_series, alternate_number, alternate_count}';
COMMENT ON COLUMN media_items.scan_information IS 'Scan information from ComicInfo.xml (scanner group, resolution, etc.)';
-- Universal fields (apply to ebooks, audiobooks, comics)
COMMENT ON COLUMN media_items.series_count IS 'Total items in series (from ComicInfo.xml Count field, or book series count)';
COMMENT ON COLUMN media_items.volume IS 'Volume/omnibus number for collected editions';
COMMENT ON COLUMN media_items.imprint IS 'Publisher imprint/subdivision (e.g., Vertigo, HarperCollinsEpic, DC Black Label)';
COMMENT ON COLUMN media_items.age_rating IS 'Age rating from metadata: Everyone, Teen, Mature, Adult (applies to all formats)';
COMMENT ON COLUMN media_items.web_url IS 'URL to info page (Goodreads, ComicVine, MangaUpdates, Audible, etc.)';
-- Additional metadata (applies to all formats)
COMMENT ON COLUMN media_items.metadata_notes IS 'Notes from metadata files (ComicInfo.xml, EPUB, PDF) - distinct from user notes in media_notes table';
COMMENT ON COLUMN media_items.community_rating IS 'Pre-existing community rating from metadata files (scale 0.0-10.0, DOUBLE PRECISION) - distinct from user ratings in media_ratings table';
COMMENT ON COLUMN media_items.summary IS 'Summary from ComicInfo.xml (may be merged with description from Calibre)';
```
### 1.4 Regenerate Database Models
**Command:**
```bash
cd internal/database && sqlc generate
```
**Expected Changes:**
- `CreateMediaItemParams` struct will include `MangaType` and `ReadingDirection` fields
- `UpdateMediaItemIdentifiersParams` struct may need updating
- All queries that return `MediaItems` will include new columns
---
## Phase 2: Update Data Structures
### 2.1 Enhance `ComicInfo` Struct
**File:** `internal/services/media_scanner.go`
**Location:** Find the struct starting with:
```go
// ComicInfo represents metadata from ComicInfo.xml
type ComicInfo struct {
```
**Current Code:**
```go
// ComicInfo represents metadata from ComicInfo.xml
type ComicInfo struct {
XMLName xml.Name `xml:"ComicInfo"`
Title string `xml:"Title"`
Series string `xml:"Series"`
Number int `xml:"Number"`
Volume int `xml:"Volume"`
Publisher string `xml:"Publisher"`
Year int `xml:"Year"`
Month int `xml:"Month"`
Day int `xml:"Day"`
Writer string `xml:"Writer"`
Penciller string `xml:"Penciller"`
Inker string `xml:"Inker"`
Colorist string `xml:"Colorist"`
Letterer string `xml:"Letterer"`
CoverArtist string `xml:"CoverArtist"`
Genre string `xml:"Genre"`
Tags string `xml:"Tags"`
Web string `xml:"Web"`
Notes string `xml:"Notes"`
}
```
**Updated Code:**
```go
type ComicInfo struct {
XMLName xml.Name `xml:"ComicInfo"`
// Basic metadata (already extracted)
Title string `xml:"Title"`
Series string `xml:"Series"`
Number int `xml:"Number"`
Volume int `xml:"Volume"`
Publisher string `xml:"Publisher"`
Year int `xml:"Year"`
Month int `xml:"Month"`
Day int `xml:"Day"`
Writer string `xml:"Writer"`
Penciller string `xml:"Penciller"`
Inker string `xml:"Inker"`
Colorist string `xml:"Colorist"`
Letterer string `xml:"Letterer"`
CoverArtist string `xml:"CoverArtist"`
Genre string `xml:"Genre"`
Tags string `xml:"Tags"`
Web string `xml:"Web"`
Notes string `xml:"Notes"`
// NEW: Reading direction fields from ComicInfo.xml v2.0
Manga string `xml:"Manga"` // Unknown, No, Yes, YesAndRightToLeft
LanguageISO string `xml:"LanguageISO"` // ISO 639-1 language code for heuristics
// NEW: Additional comic-specific fields (19 total fields from ComicInfo.xml)
Count int `xml:"Count"` // Total issues in series
AlternateSeries string `xml:"AlternateSeries"`
AlternateNumber int `xml:"AlternateNumber"`
AlternateCount int `xml:"AlternateCount"`
Summary string `xml:"Summary"`
Imprint string `xml:"Imprint"`
StoryArc string `xml:"StoryArc"`
SeriesGroup string `xml:"SeriesGroup"`
AgeRating string `xml:"AgeRating"`
CommunityRating float64 `xml:"CommunityRating"`
MainCharacterOrTeam string `xml:"MainCharacterOrTeam"`
Review string `xml:"Review"`
BlackAndWhite string `xml:"BlackAndWhite"` // "Yes" or "No"
ScanInformation string `xml:"ScanInformation"`
Characters string `xml:"Characters"`
Teams string `xml:"Teams"`
Locations string `xml:"Locations"`
}
```
### 2.2 Enhance `MediaMetadata` Struct
**File:** `internal/services/media_scanner.go`
**Location:** Find the struct starting with:
```go
// MediaMetadata contains extracted metadata for media files (ebooks, comics, manga)
type MediaMetadata struct {
```
**Current Code:**
```go
// MediaMetadata contains extracted metadata for media files (ebooks, comics, manga)
type MediaMetadata struct {
Title string
Author string
Description string
Series string
SeriesNumber int32
Publisher string
PublishDate time.Time
Contributors []string
CoverPath string
ISBN string
ASIN string
Tags []string
FileHashInfo *HashInfo
FileFormats []*FormatInfo
}
```
**Updated Code:**
```go
// MediaMetadata contains extracted metadata for media files (ebooks, comics, manga)
type MediaMetadata struct {
// Existing fields
Title string
Author string
Description string
Series string
SeriesNumber int32
Publisher string
PublishDate time.Time
Contributors []string
CoverPath string
ISBN string
ASIN string
Tags []string
FileHashInfo *HashInfo
FileFormats []*FormatInfo
// NEW: Reading direction fields (from ComicInfo.xml or computed)
MangaType string // Raw ComicInfo.xml Manga field
ReadingDirection string // Computed: auto, ltr, rtl, vertical
Language string // ISO 639-1 language code
// NEW: Additional metadata fields (from ComicInfo.xml or other metadata sources)
// Universal fields (apply to ebooks, audiobooks, comics)
SeriesCount int32 // Total items in series (Count field for comics, series count for books)
Volume int32 // Volume/omnibus number
Imprint string // Publisher imprint
AgeRating string // Age rating (Everyone, Teen, Mature, Adult)
WebURL string // URL to info page (Goodreads, ComicVine, etc.)
MetadataNotes string // Notes from metadata files (not user notes)
CommunityRating float64 // Pre-existing community rating (0.0-10.0) - maps to DOUBLE PRECISION in database
// Comic-specific fields
StoryArc string // Story arc name
IsBlackAndWhite bool // Black and white flag
AlternateInfo string // JSONB string of alternate series info
ScanInformation string // Scan information
Summary string // Summary from ComicInfo.xml
}
```
---
## Phase 3: Smart Metadata Merging Logic
### 3.1 Create Metadata Merge Function
**File:** `internal/services/media_scanner.go`
**Location:** Insert this new function after `extractCalibreSidecar()` function. Find the function ending with:
```go
return metadata
}
```
and before the next function. The new `mergeMetadata()` function should be inserted here.
**New Function:**
```go
// mergeMetadata intelligently merges metadata from multiple sources
// Priority: metadata.opf (Calibre) → embedded metadata → folder structure → filename
// For comics: metadata.opf → ComicInfo.xml → folder structure → filename
func (s *MediaScanner) mergeMetadata(path string, calibreMetadata *MediaMetadata) (*MediaMetadata, error) {
metadata := calibreMetadata
if metadata == nil {
metadata = &MediaMetadata{}
}
ext := strings.ToLower(filepath.Ext(path))
// For comic archives, try to extract ComicInfo.xml
if ext == ".cbz" || ext == ".cbr" || ext == ".cb7" || ext == ".cbt" {
comicInfo, cover, err := extractComicMetadata(path)
if err != nil {
fmt.Printf("Warning: failed to extract comic metadata from %s: %v\n", path, err)
} else if comicInfo != nil {
// Merge ComicInfo.xml fields (only if not already set from Calibre)
if metadata.Title == "" && comicInfo.Title != "" {
metadata.Title = comicInfo.Title
}
if metadata.Series == "" && comicInfo.Series != "" {
metadata.Series = comicInfo.Series
}
if metadata.SeriesNumber == 0 && comicInfo.Number > 0 {
metadata.SeriesNumber = int32(comicInfo.Number)
}
if metadata.Publisher == "" && comicInfo.Publisher != "" {
metadata.Publisher = comicInfo.Publisher
}
if metadata.Author == "" && comicInfo.Writer != "" {
metadata.Author = comicInfo.Writer
}
if metadata.Description == "" && comicInfo.Summary != "" {
metadata.Description = comicInfo.Summary
}
// NEW: Always extract reading direction from ComicInfo.xml
// (even if metadata.opf exists, since Calibre doesn't support this field)
metadata.MangaType = normalizeMangaType(comicInfo.Manga)
metadata.ReadingDirection = determineReadingDirection(comicInfo)
metadata.Language = comicInfo.LanguageISO
// NEW: Extract additional comic-specific fields
// Series information
if metadata.SeriesCount == 0 && comicInfo.Count > 0 {
metadata.SeriesCount = int32(comicInfo.Count)
}
if metadata.Volume == 0 && comicInfo.Volume > 0 {
metadata.Volume = int32(comicInfo.Volume)
}
// Publisher and classification
if metadata.Imprint == "" && comicInfo.Imprint != "" {
metadata.Imprint = comicInfo.Imprint
}
if metadata.StoryArc == "" && comicInfo.StoryArc != "" {
metadata.StoryArc = comicInfo.StoryArc
}
if metadata.AgeRating == "" && comicInfo.AgeRating != "" {
metadata.AgeRating = normalizeAgeRating(comicInfo.AgeRating)
}
// NEW: Process genres and tags (universal logic for all formats)
// Extract genre tags from ComicInfo.xml (Genre + Tags + Characters + Teams + Locations)
genreTags := extractGenreTagsFromComicInfo(comicInfo)
processGenresAndTags(metadata, genreTags)
// Additional metadata
if metadata.WebURL == "" && comicInfo.Web != "" {
metadata.WebURL = comicInfo.Web
}
if metadata.MetadataNotes == "" && comicInfo.Notes != "" {
metadata.MetadataNotes = comicInfo.Notes
}
if metadata.ScanInformation == "" && comicInfo.ScanInformation != "" {
metadata.ScanInformation = comicInfo.ScanInformation
}
if metadata.Summary == "" && comicInfo.Summary != "" {
metadata.Summary = comicInfo.Summary
}
// Boolean fields
if !metadata.IsBlackAndWhite && strings.ToLower(comicInfo.BlackAndWhite) == "yes" {
metadata.IsBlackAndWhite = true
}
if metadata.CommunityRating == 0 && comicInfo.CommunityRating > 0 {
metadata.CommunityRating = comicInfo.CommunityRating
}
// Alternate series information (store as JSONB string)
if metadata.AlternateInfo == "" && (comicInfo.AlternateSeries != "" || comicInfo.AlternateNumber > 0) {
alternateData := map[string]interface{}{}
if comicInfo.AlternateSeries != "" {
alternateData["alternate_series"] = comicInfo.AlternateSeries
}
if comicInfo.AlternateNumber > 0 {
alternateData["alternate_number"] = comicInfo.AlternateNumber
}
if comicInfo.AlternateCount > 0 {
alternateData["alternate_count"] = comicInfo.AlternateCount
}
if len(alternateData) > 0 {
jsonBytes, err := json.Marshal(alternateData)
if err == nil {
metadata.AlternateInfo = string(jsonBytes)
}
}
}
// REMOVED: Tag enhancement now handled by processGenresAndTags()
// Characters, Teams, Locations are already processed via extractGenreTagsFromComicInfo()
// Extract cover if not already present
if len(cover) > 0 && metadata.CoverPath == "" {
coverPath := path + ".cover.jpg"
if err := os.WriteFile(coverPath, cover, 0644); err == nil {
metadata.CoverPath = s.getRelativePath(coverPath)
}
}
fmt.Printf("Merged comic metadata from %s: title=%s, series=%s, issue=%d, manga=%s, direction=%s\n",
path, comicInfo.Title, comicInfo.Series, comicInfo.Number, comicInfo.Manga, metadata.ReadingDirection)
}
}
return metadata, nil
}
// containsTag checks if a tag already exists in the tags array
func containsTag(tags []string, tag string) bool {
tag = strings.ToLower(tag)
for _, t := range tags {
if strings.ToLower(t) == tag {
return true
}
}
return false
}
// normalizeAgeRating normalizes age rating from ComicInfo.xml to standard values
func normalizeAgeRating(rating string) string {
rating = strings.ToLower(strings.TrimSpace(rating))
switch rating {
case "everyone", "e", "all ages":
return "Everyone"
case "teen", "t", "13+", "13+up":
return "Teen"
case "mature", "m", "17+", "17+up", "adults only":
return "Mature"
case "adult", "a", "18+":
return "Adult"
default:
return rating // Return original if unknown
}
}
```
### 3.2 Create Reading Direction Helper Functions
**File:** `internal/services/media_scanner.go`
**Location:** After `mergeMetadata()` function
**New Functions:**
```go
// normalizeMangaType normalizes ComicInfo.xml Manga field to database enum values
func normalizeMangaType(manga string) string {
switch strings.ToLower(strings.ReplaceAll(manga, " ", "")) {
case "unknown":
return "unknown"
case "no":
return "no"
case "yes":
return "yes"
case "yesandrighttoleft":
return "yes_and_right_to_left"
default:
return "unknown"
}
}
// determineReadingDirection computes reading direction from ComicInfo metadata
// Uses Manga field + language heuristics + genre tags
func determineReadingDirection(comicInfo *ComicInfo) string {
// 1. Check explicit Manga field
manga := normalizeMangaType(comicInfo.Manga)
switch manga {
case "yes_and_right_to_left":
return "rtl" // Traditional Japanese manga
case "yes", "no":
return "ltr" // Manga style but LTR, or Western comic
}
// 2. Language heuristic: Japanese → RTL
lang := strings.ToLower(comicInfo.LanguageISO)
if lang == "ja" || lang == "jpn" {
return "rtl"
}
// 3. Genre heuristic: webtoons/manhwa → vertical
tags := strings.ToLower(comicInfo.Tags + " " + comicInfo.Genre)
if strings.Contains(tags, "webtoon") || strings.Contains(tags, "manhwa") {
return "vertical" // Korean/Chinese webcomics
}
if strings.Contains(tags, "manga") && (lang == "ja" || lang == "jpn") {
return "rtl" // Japanese manga
}
// 4. Default: LTR (Western comics)
return "ltr"
}
```
### 3.3 Genre and Tag Processing Logic (Universal for ALL Formats)
**File:** `internal/services/media_scanner.go`
**Location:** After `normalizeAgeRating()` function
**New Function:**
```go
// processGenresAndTags ensures ALL genres appear in the tags array without duplication
// This applies to ALL formats: EPUB, ComicInfo.xml, PDF metadata
// Strategy: Use existing `genre` column for primary genre, `tags` array for all genres
func processGenresAndTags(metadata *MediaMetadata, genreTags []string) {
if metadata.Tags == nil {
metadata.Tags = []string{}
}
// 1. Set primary genre (first genre tag wins if not already set)
if metadata.Genre == "" && len(genreTags) > 0 {
metadata.Genre = genreTags[0]
}
// 2. Ensure ALL genre tags appear in tags array (without duplication)
for _, genreTag := range genreTags {
genreTag = strings.TrimSpace(genreTag)
if genreTag != "" && !containsTag(metadata.Tags, genreTag) {
metadata.Tags = append(metadata.Tags, genreTag)
}
}
}
// extractGenreTagsFromEPUB extracts all <dc:subject> values from EPUB
// Returns array of genre tags
func extractGenreTagsFromEPUB(book *epub.Book) []string {
var genreTags []string
// EPUB stores genres in <dc:subject> metadata
if subjects, err := book.MetadataByKey("subject"); err == nil && len(subjects) > 0 {
for _, subject := range subjects {
subject = strings.TrimSpace(subject)
if subject != "" {
genreTags = append(genreTags, subject)
}
}
}
return genreTags
}
// extractGenreTagsFromComicInfo extracts genres from ComicInfo.xml
// Genre field + Tags field + Characters + Teams + Locations
// Returns array of genre tags
func extractGenreTagsFromComicInfo(comicInfo *ComicInfo) []string {
var genreTags []string
// 1. Add Genre field
if comicInfo.Genre != "" {
genreTags = append(genreTags, strings.Split(comicInfo.Genre, ",")...)
}
// 2. Add Tags field (comma-separated)
if comicInfo.Tags != "" {
genreTags = append(genreTags, strings.Split(comicInfo.Tags, ",")...)
}
// 3. Add Characters (comma-separated)
if comicInfo.Characters != "" {
genreTags = append(genreTags, strings.Split(comicInfo.Characters, ",")...)
}
// 4. Add Teams (comma-separated)
if comicInfo.Teams != "" {
genreTags = append(genreTags, strings.Split(comicInfo.Teams, ",")...)
}
// 5. Add Locations (comma-separated)
if comicInfo.Locations != "" {
genreTags = append(genreTags, strings.Split(comicInfo.Locations, ",")...)
}
// Trim whitespace from all tags
for i := range genreTags {
genreTags[i] = strings.TrimSpace(genreTags[i])
}
return genreTags
}
```
**Usage in mergeMetadata():**
```go
// In mergeMetadata() function, after extracting metadata:
// For EPUB files
if ext == ".epub" {
book, err := epub.ReadBook(path)
if err == nil {
genreTags := extractGenreTagsFromEPUB(book)
processGenresAndTags(metadata, genreTags)
}
}
// For comic archives
if ext == ".cbz" || ext == ".cbr" || ext == ".cb7" || ext == ".cbt" {
comicInfo, cover, err := extractComicMetadata(path)
if err == nil && comicInfo != nil {
genreTags := extractGenreTagsFromComicInfo(comicInfo)
processGenresAndTags(metadata, genreTags)
}
}
```
**Key Points:**
1. **Single `genre` column**: Primary classification (first genre tag)
2. **`tags` array**: ALL genres/tags without duplication
3. **Universal logic**: Works the same for EPUB, ComicInfo.xml, PDF metadata
4. **No `genre_comic` column needed**: Reuse existing `genre` column
5. **Deduplication**: `containsTag()` helper prevents duplicates
**Examples:**
**EPUB:**
- Input: `<dc:subject>Fantasy</dc:subject><dc:subject>Adventure</dc:subject>`
- Result: `genre` = "Fantasy", `tags` = ["Fantasy", "Adventure"]
**ComicInfo.xml:**
- Input: `<Genre>Action</Genre><Tags>Adventure, Fighting</Tags><Characters>Naruto, Sasuke</Characters>`
- Result: `genre` = "Action", `tags` = ["Action", "Adventure", "Fighting", "Naruto", "Sasuke"]
**Calibre metadata.opf:**
- Input: Multiple `<dc:subject>` tags
- Result: `genre` = first subject, `tags` = all subjects
### 3.4 Update `extractMetadata()` Function
**File:** `internal/services/media_scanner.go`
**Location:** Find the function starting with:
```go
func (s *MediaScanner) extractMetadata(path string) (*MediaMetadata, error) {
```
**Current Code (first ~15 lines of the function):**
```go
func (s *MediaScanner) extractMetadata(path string) (*MediaMetadata, error) {
// NEW: Try Calibre sidecar first
if metadata := s.extractCalibreSidecar(path); metadata != nil {
fmt.Printf("Using Calibre metadata.opf for %s\n", path)
// Try to find cover image for sidecar metadata
coverPath := findSidecarCover(path)
if coverPath != "" {
metadata.CoverPath = s.getRelativePath(coverPath)
}
return metadata, nil
}
// EXISTING: Fallback to embedded metadata
ext := strings.ToLower(filepath.Ext(path))
...
```
**Updated Code:**
```go
func (s *MediaScanner) extractMetadata(path string) (*MediaMetadata, error) {
// Try Calibre sidecar first
calibreMetadata := s.extractCalibreSidecar(path)
if calibreMetadata != nil {
fmt.Printf("Using Calibre metadata.opf for %s\n", path)
// Try to find cover image for sidecar metadata
coverPath := findSidecarCover(path)
if coverPath != "" {
calibreMetadata.CoverPath = s.getRelativePath(coverPath)
}
}
// NEW: Smart merge - also parse embedded metadata for missing fields
// This ensures we get reading direction from ComicInfo.xml even when metadata.opf exists
return s.mergeMetadata(path, calibreMetadata)
}
```
**Key Changes:**
- Rename local variable from `metadata` to `calibreMetadata` for clarity
- Remove early return when Calibre metadata exists
- Call `mergeMetadata()` to merge Calibre + embedded metadata
- Remove the fallback switch statement (`.epub`, `.pdf`, etc.) - this logic moves into `mergeMetadata()`
---
## Phase 4: Update Media Item Creation
### 4.1 Update CreateMediaItem SQL Query
**File:** `internal/database/queries/queries.sql`
**Location:** Find the section starting with:
```sql
-- name: CreateMediaItem :one
INSERT INTO media_items (library_id, title, author, isbn...
```
and ending with:
```sql
...added_by_admin_id, created_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28)
RETURNING *;
```
**Current Query:**
```sql
-- name: CreateMediaItem :one
INSERT INTO media_items (library_id, title, author, isbn, description, file_path, file_size, mime_type, cover_image_path, series, series_number, tags, tags_search, asin, date_published, publisher, contributors, contributors_search, language, edition, page_count, genre, copyright_year, goodreads_id, openlibrary_id, google_books_id, added_by_admin_id, created_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28)
RETURNING *;
```
**Updated Query:**
```sql
-- name: CreateMediaItem :one
INSERT INTO media_items (library_id, title, author, isbn, description, file_path, file_size, mime_type, cover_image_path, series, series_number, tags, tags_search, asin, date_published, publisher, contributors, contributors_search, language, edition, page_count, genre, copyright_year, goodreads_id, openlibrary_id, google_books_id, added_by_admin_id, created_at, manga_type, reading_direction, series_count, volume, imprint, age_rating, web_url, story_arc, is_black_and_white, metadata_notes, community_rating, alternate_info, scan_information, summary)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35, $36, $37, $38, $39, $40, $41, $42)
RETURNING *;
```
**Changes:**
- Added 14 new columns to INSERT: `manga_type, reading_direction, series_count, volume, imprint, age_rating, web_url, story_arc, is_black_and_white, metadata_notes, community_rating, alternate_info, scan_information, summary`
- Added 14 new VALUES parameters: `$29-$42` (note: count is 14 new parameters, not 15 as previously stated)
### 4.2 Regenerate sqlc Models
**Command:**
```bash
cd internal/database && sqlc generate
```
**Expected Changes:**
- `CreateMediaItemParams` struct will include 14 new fields
- `MediaItems.CommunityRating` will be `pgtype.Float8` (not pgtype.Numeric)
- All SELECT/RETURNING queries will include the new columns
**Important:** After running `sqlc generate`, verify that `CommunityRating` in the generated `MediaItems` struct is `pgtype.Float8`, not `pgtype.Numeric`.
### 4.3 Update CreateMediaItem Call
**File:** `internal/services/media_scanner.go`
**Location:** Find this section in `processMediaFile()` function:
```go
// Create media item in database
relativePath := s.getRelativePath(path)
createdItem, err := s.db.CreateMediaItem(ctx, database.CreateMediaItemParams{
LibraryID: libraryID,
Title: metadata.Title,
Author: pgtype.Text{String: metadata.Author, Valid: metadata.Author != ""},
Isbn: pgtype.Text{String: utils.NormalizeISBNSafe(metadata.ISBN), Valid: metadata.ISBN != ""},
...
AddedByAdminID: s.adminID,
CreatedAt: pgtype.Timestamptz{Time: fileModTime, Valid: true},
})
```
**Add to CreateMediaItem call (insert before the closing `}`):**
```go
// Create media item in database
relativePath := s.getRelativePath(path)
createdItem, err := s.db.CreateMediaItem(ctx, database.CreateMediaItemParams{
LibraryID: libraryID,
Title: metadata.Title,
Author: pgtype.Text{String: metadata.Author, Valid: metadata.Author != ""},
Isbn: pgtype.Text{String: utils.NormalizeISBNSafe(metadata.ISBN), Valid: metadata.ISBN != ""},
Asin: pgtype.Text{String: metadata.ASIN, Valid: metadata.ASIN != ""},
Description: pgtype.Text{String: metadata.Description, Valid: metadata.Description != ""},
FilePath: relativePath,
FileSize: pgtype.Int8{Int64: info.Size(), Valid: true},
MimeType: pgtype.Text{String: s.getMimeType(path), Valid: true},
CoverImagePath: pgtype.Text{String: metadata.CoverPath, Valid: metadata.CoverPath != ""},
Series: pgtype.Text{String: metadata.Series, Valid: metadata.Series != ""},
SeriesNumber: pgtype.Int4{Int32: metadata.SeriesNumber, Valid: metadata.SeriesNumber > 0},
Publisher: pgtype.Text{String: metadata.Publisher, Valid: metadata.Publisher != ""},
DatePublished: pgtype.Date{Time: metadata.PublishDate, Valid: !metadata.PublishDate.IsZero()},
Contributors: metadata.Contributors,
ContributorsSearch: contributorsSearch,
Tags: metadata.Tags,
TagsSearch: tagsSearch,
AddedByAdminID: s.adminID,
CreatedAt: pgtype.Timestamptz{Time: fileModTime, Valid: true},
// NEW: Reading direction and comic metadata fields
MangaType: pgtype.Text{String: metadata.MangaType, Valid: metadata.MangaType != ""},
ReadingDirection: pgtype.Text{String: metadata.ReadingDirection, Valid: metadata.ReadingDirection != ""},
SeriesCount: pgtype.Int4{Int32: metadata.SeriesCount, Valid: metadata.SeriesCount > 0},
Volume: pgtype.Int4{Int32: metadata.Volume, Valid: metadata.Volume > 0},
Imprint: pgtype.Text{String: metadata.Imprint, Valid: metadata.Imprint != ""},
AgeRating: pgtype.Text{String: metadata.AgeRating, Valid: metadata.AgeRating != ""},
WebUrl: pgtype.Text{String: metadata.WebURL, Valid: metadata.WebURL != ""},
StoryArc: pgtype.Text{String: metadata.StoryArc, Valid: metadata.StoryArc != ""},
IsBlackAndWhite: pgtype.Bool{Bool: metadata.IsBlackAndWhite, Valid: true},
MetadataNotes: pgtype.Text{String: metadata.MetadataNotes, Valid: metadata.MetadataNotes != ""},
AlternateInfo: func() []byte {
if metadata.AlternateInfo != "" {
return []byte(metadata.AlternateInfo)
}
return nil
}(),
ScanInformation: pgtype.Text{String: metadata.ScanInformation, Valid: metadata.ScanInformation != ""},
Summary: pgtype.Text{String: metadata.Summary, Valid: metadata.Summary != ""},
CommunityRating: pgtype.Float8{Float64: metadata.CommunityRating, Valid: metadata.CommunityRating > 0},
})
```
**Important Type Conversions:**
- `AlternateInfo`: `string``[]byte` (JSONB in database)
- `CommunityRating`: `float64` (0-10 scale) → `pgtype.Float8` (DOUBLE PRECISION) - simple direct assignment
- `WebUrl`: Field name is `WebUrl` (camelCase) not `WebURL`
### 4.4 Remove Duplicate Comic Metadata Extraction
**File:** `internal/services/media_scanner.go`
**Location:** In `processMediaFile()` function, find this section:
```go
var comicInfo *ComicInfo
var coverImage []byte
ext := strings.ToLower(filepath.Ext(path))
if ext == ".cbz" || ext == ".cbr" || ext == ".cb7" || ext == ".cbt" ||
strings.HasSuffix(strings.ToLower(path), ".tar.gz") ||
strings.HasSuffix(strings.ToLower(path), ".tar.bz2") ||
strings.HasSuffix(strings.ToLower(path), ".tgz") ||
strings.HasSuffix(strings.ToLower(path), ".tbz2") {
info, cover, err := extractComicMetadata(path)
if err != nil {
fmt.Printf("Warning: failed to extract comic metadata from %s: %v\n", path, err)
} else {
comicInfo = info
coverImage = cover
if comicInfo.Title != "" && metadata.Title == "" {
metadata.Title = comicInfo.Title
}
if comicInfo.Series != "" && metadata.Series == "" {
metadata.Series = comicInfo.Series
}
if comicInfo.Number > 0 && metadata.SeriesNumber == 0 {
metadata.SeriesNumber = int32(comicInfo.Number)
}
if comicInfo.Publisher != "" && metadata.Publisher == "" {
metadata.Publisher = comicInfo.Publisher
}
if comicInfo.Writer != "" && metadata.Author == "" {
metadata.Author = comicInfo.Writer
}
if len(coverImage) > 0 && metadata.CoverPath == "" {
coverPath := path + ".cover.jpg"
if err := os.WriteFile(coverPath, coverImage, 0644); err == nil {
metadata.CoverPath = s.getRelativePath(coverPath)
}
}
fmt.Printf("Extracted comic metadata from %s: title=%s, series=%s, issue=%d\n",
path, comicInfo.Title, comicInfo.Series, comicInfo.Number)
}
}
```
**Remove this entire block** and replace with:
```go
// REMOVED: Comic metadata extraction now handled by mergeMetadata()
// This avoids duplicate extraction and ensures smart merging happens
```
**Reason:** This block was extracting comic metadata and merging fields manually. The new `mergeMetadata()` function handles this more comprehensively with all 29 ComicInfo.xml fields.
---
## Phase 5: API Layer Updates
### 5.1 Update TypeScript API Types
**File:** `web/src/types/api.d.ts`
**Location:** Find the interface starting with:
```typescript
interface MediaItemSummary {
id: string;
library_id: string;
title: string;
...
library_name: string;
library_type_name: string;
}
```
**Current Code:**
```typescript
interface MediaItemSummary {
id: string;
library_id: string;
title: string;
author?: string;
isbn?: string;
description?: string;
file_path: string;
file_size?: number;
mime_type?: string;
cover_image_path?: string;
series?: string;
series_number?: number;
tags?: string[];
asin?: string;
date_published?: string;
publisher?: string;
contributors?: string[];
language?: string;
edition?: string;
page_count?: number;
genre?: string;
copyright_year?: number;
goodreads_id?: string;
openlibrary_id?: string;
google_books_id?: string;
added_by_admin_id?: string;
created_at: string;
updated_at: string;
format_group: string;
format_mimetype?: string;
is_reflowable?: boolean;
has_fixed_layout?: boolean;
total_characters?: number;
chapter_count?: number;
entitlement_id?: string;
revision_number?: number;
kobo_content_id?: string;
kobo_metadata?: string;
tags_search?: string[];
contributors_search?: string[];
file_sha256?: string;
opf_identifier?: string;
opf_uuid?: string;
hash_confidence?: string;
library_name: string;
library_type_name: string;
}
```
**Updated Code:**
```typescript
interface MediaItemSummary {
id: string;
library_id: string;
title: string;
author?: string;
isbn?: string;
description?: string;
file_path: string;
file_size?: number;
mime_type?: string;
cover_image_path?: string;
series?: string;
series_number?: number;
tags?: string[];
asin?: string;
date_published?: string;
publisher?: string;
contributors?: string[];
language?: string;
edition?: string;
page_count?: number;
genre?: string;
copyright_year?: number;
goodreads_id?: string;
openlibrary_id?: string;
google_books_id?: string;
added_by_admin_id?: string;
created_at: string;
updated_at: string;
format_group: string;
format_mimetype?: string;
is_reflowable?: boolean;
has_fixed_layout?: boolean;
total_characters?: number;
chapter_count?: number;
entitlement_id?: string;
revision_number?: number;
kobo_content_id?: string;
kobo_metadata?: string;
tags_search?: string[];
contributors_search?: string[];
file_sha256?: string;
opf_identifier?: string;
opf_uuid?: string;
hash_confidence?: string;
library_name: string;
library_type_name: string;
// NEW: Reading direction fields for manga/comics
manga_type?: "unknown" | "no" | "yes" | "yes_and_right_to_left";
reading_direction?: "auto" | "ltr" | "rtl" | "vertical";
// NEW: Universal metadata fields (apply to ebooks, audiobooks, comics)
series_count?: number;
volume?: number;
imprint?: string;
age_rating?: string; // 'Everyone' | 'Teen' | 'Mature' | 'Adult'
web_url?: string;
metadata_notes?: string;
community_rating?: number;
// NEW: Comic-specific fields
story_arc?: string;
is_black_and_white?: boolean;
alternate_info?: {
alternate_series?: string;
alternate_number?: number;
alternate_count?: number;
};
scan_information?: string;
summary?: string;
}
```
**Add these fields at the end of the interface, just before the closing `}`.**
### 5.2 Verify API Response Handling
**Files to Check:**
- `internal/handlers/media.go` - Ensure `SearchMediaItems()` includes new columns
- `internal/handlers/collections.go` - Ensure `BookInfo` can include reading direction if needed
- `internal/database/queries.sql` - Ensure queries select `manga_type` and `reading_direction`
**Note:** SQL queries should automatically include new columns after `sqlc generate` is run.
---
## Phase 6: Testing
### 6.1 Unit Tests
**File:** `internal/services/media_scanner_comic_test.go`
**New Test Cases to Add:**
```go
func TestNormalizeMangaType(t *testing.T) {
tests := []struct {
name string
input string
expected string
}{
{"Unknown", "Unknown", "unknown"},
{"No", "No", "no"},
{"Yes", "Yes", "yes"},
{"YesAndRightToLeft", "YesAndRightToLeft", "yes_and_right_to_left"},
{"Lowercase yesandrighttoleft", "yesandrighttoleft", "yes_and_right_to_left"},
{"With spaces", "Yes And Right To Left", "yes_and_right_to_left"},
{"Invalid", "invalid", "unknown"},
{"Empty", "", "unknown"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := normalizeMangaType(tt.input)
if result != tt.expected {
t.Errorf("normalizeMangaType(%q) = %q; want %q", tt.input, result, tt.expected)
}
})
}
}
func TestDetermineReadingDirection(t *testing.T) {
tests := []struct {
name string
manga string
language string
tags string
genre string
expected string
}{
{"Explicit RTL", "YesAndRightToLeft", "en", "", "", "rtl"},
{"Explicit LTR (Yes)", "Yes", "ja", "", "", "ltr"},
{"Explicit LTR (No)", "No", "en", "", "", "ltr"},
{"Japanese heuristic", "Unknown", "ja", "", "", "rtl"},
{"Japanese with full code", "Unknown", "jpn", "", "", "rtl"},
{"Webtoon Korean", "Unknown", "ko", "Webtoon", "", "vertical"},
{"Manhwa in tags", "Unknown", "ko", "", "Manhwa", "vertical"},
{"Manga + Japanese", "Unknown", "ja", "Manga", "", "rtl"},
{"Western default", "Unknown", "en", "", "", "ltr"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
comicInfo := &ComicInfo{
Manga: tt.manga,
LanguageISO: tt.language,
Tags: tt.tags,
Genre: tt.genre,
}
result := determineReadingDirection(comicInfo)
if result != tt.expected {
t.Errorf("determineReadingDirection() = %q; want %q", result, tt.expected)
}
})
}
}
func TestMergeMetadata(t *testing.T) {
// Test case 1: metadata.opf exists, ComicInfo.xml exists
// Expected: metadata.opf fields take priority, except reading_direction from ComicInfo
// Test case 2: metadata.opf exists, ComicInfo.xml missing
// Expected: only metadata.opf fields used
// Test case 3: metadata.opf missing, ComicInfo.xml exists
// Expected: ComicInfo.xml fields used
// Test case 4: Both missing
// Expected: fallback to filename
}
```
### 6.2 Integration Tests
**File:** `cmd/server/tests/comic_metadata_test.go`
**Create New File:** Following test_helpers pattern
**Test Structure:**
```go
package tests
import (
"context"
"testing"
"bookhoard/internal/database"
"github.com/jackc/pgx/v5/pgtype"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestComicMetadataExtraction tests that comic metadata fields are stored correctly
func TestComicMetadataExtraction(t *testing.T) {
setup := setupDeviceTest(t)
defer setup.Server.Close()
ctx := context.Background()
libraryID := setup.CreateLibrary(t, "Comic Test Library", "comic")
t.Run("CBZ with RTL manga", func(t *testing.T) {
// Insert test media item with full comic metadata
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
LibraryID: libraryID,
Title: "Test Manga",
FilePath: "/test/manga.cbz",
MangaType: pgtype.Text{String: "yes_and_right_to_left", Valid: true},
ReadingDirection: pgtype.Text{String: "rtl", Valid: true},
SeriesCount: pgtype.Int4{Int32: 12, Valid: true},
Volume: pgtype.Int4{Int32: 1, Valid: true},
StoryArc: pgtype.Text{String: "The Dark Phoenix Saga", Valid: true},
AgeRating: pgtype.Text{String: "Teen", Valid: true},
CommunityRating: pgtype.Float8{Float64: 8.5, Valid: true},
Imprint: pgtype.Text{String: "Shonen Jump", Valid: true},
IsBlackAndWhite: pgtype.Bool{Bool: false, Valid: true},
})
require.NoError(t, err)
// Query it back
items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
LibraryID: libraryID,
})
require.NoError(t, err)
require.Greater(t, len(items), 0)
item := items[0]
assert.Equal(t, "Test Manga", item.Title)
assert.Equal(t, "yes_and_right_to_left", item.MangaType.String)
assert.Equal(t, "rtl", item.ReadingDirection.String)
assert.Equal(t, int32(12), item.SeriesCount.Int32)
assert.Equal(t, int32(1), item.Volume.Int32)
assert.Equal(t, "The Dark Phoenix Saga", item.StoryArc.String)
assert.Equal(t, "Teen", item.AgeRating.String)
assert.Equal(t, 8.5, item.CommunityRating.Float64)
assert.Equal(t, "Shonen Jump", item.Imprint.String)
assert.False(t, item.IsBlackAndWhite.Bool)
})
t.Run("CBZ with Western comic", func(t *testing.T) {
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
LibraryID: libraryID,
Title: "Test Comic",
FilePath: "/test/comic.cbz",
MangaType: pgtype.Text{String: "no", Valid: true},
ReadingDirection: pgtype.Text{String: "ltr", Valid: true},
Imprint: pgtype.Text{String: "Vertigo", Valid: true},
IsBlackAndWhite: pgtype.Bool{Bool: true, Valid: true},
StoryArc: pgtype.Text{String: "Batman: Year One", Valid: true},
SeriesCount: pgtype.Int4{Int32: 4, Valid: true},
})
require.NoError(t, err)
items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
LibraryID: libraryID,
})
require.NoError(t, err)
item := items[0]
assert.Equal(t, "no", item.MangaType.String)
assert.Equal(t, "ltr", item.ReadingDirection.String)
assert.Equal(t, "Vertigo", item.Imprint.String)
assert.True(t, item.IsBlackAndWhite.Bool)
assert.Equal(t, "Batman: Year One", item.StoryArc.String)
assert.Equal(t, int32(4), item.SeriesCount.Int32)
})
t.Run("Comic with minimal metadata", func(t *testing.T) {
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
LibraryID: libraryID,
Title: "Minimal Comic",
FilePath: "/test/minimal.cbz",
// Only required fields - comic metadata should default appropriately
})
require.NoError(t, err)
items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
LibraryID: libraryID,
})
require.NoError(t, err)
item := items[0]
assert.Equal(t, "Minimal Comic", item.Title)
// Verify defaults
assert.Equal(t, "unknown", item.MangaType.String)
assert.Equal(t, "auto", item.ReadingDirection.String)
})
}
// TestReadingDirectionAPI tests reading direction in API responses
func TestReadingDirectionAPI(t *testing.T) {
setup := setupDeviceTest(t)
defer setup.Server.Close()
ctx := context.Background()
libraryID := setup.CreateLibrary(t, "Reading Direction Test Library", "comic")
// Create test items with different reading directions
testCases := []struct {
title string
manga string
dir string
}{
{"Japanese Manga", "yes_and_right_to_left", "rtl"},
{"Western Comic", "no", "ltr"},
{"Webtoon", "unknown", "auto"},
}
for _, tc := range testCases {
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
LibraryID: libraryID,
Title: tc.title,
FilePath: "/test/" + tc.title + ".cbz",
MangaType: pgtype.Text{String: tc.manga, Valid: true},
ReadingDirection: pgtype.Text{String: tc.dir, Valid: true},
})
require.NoError(t, err)
}
t.Run("Search API includes reading_direction", func(t *testing.T) {
items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
LibraryID: libraryID,
})
require.NoError(t, err)
require.Len(t, items, 3)
// Verify all items have reading direction set
for _, item := range items {
assert.NotEmpty(t, item.ReadingDirection.String)
assert.NotEmpty(t, item.MangaType.String)
assert.True(t, item.ReadingDirection.Valid)
}
})
t.Run("Filter by reading_direction - RTL only", func(t *testing.T) {
// Query all items and filter in-memory
items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
LibraryID: libraryID,
})
require.NoError(t, err)
// Count RTL items
rtlCount := 0
for _, item := range items {
if item.ReadingDirection.String == "rtl" {
rtlCount++
}
}
assert.Equal(t, 1, rtlCount)
})
t.Run("Verify all reading directions present", func(t *testing.T) {
items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
LibraryID: libraryID,
})
require.NoError(t, err)
directions := make(map[string]bool)
for _, item := range items {
directions[item.ReadingDirection.String] = true
}
assert.True(t, directions["rtl"])
assert.True(t, directions["ltr"])
assert.True(t, directions["auto"])
})
}
// TestUniversalMetadataFields tests universal fields apply to all formats
func TestUniversalMetadataFields(t *testing.T) {
setup := setupDeviceTest(t)
defer setup.Server.Close()
ctx := context.Background()
// Test with both comic and ebook libraries
comicLibraryID := setup.CreateLibrary(t, "Comic Library", "comic")
ebookLibraryID := setup.CreateLibrary(t, "Ebook Library", "ebook")
t.Run("Comic with universal fields", func(t *testing.T) {
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
LibraryID: comicLibraryID,
Title: "Comic with Universal Metadata",
FilePath: "/test/comic.cbz",
SeriesCount: pgtype.Int4{Int32: 10, Valid: true},
Volume: pgtype.Int4{Int32: 2, Valid: true},
Imprint: pgtype.Text{String: "DC Black Label", Valid: true},
AgeRating: pgtype.Text{String: "Mature", Valid: true},
WebURL: pgtype.Text{String: "https://example.com/comic", Valid: true},
CommunityRating: pgtype.Float8{Float64: 9.2, Valid: true},
})
require.NoError(t, err)
items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
LibraryID: comicLibraryID,
})
require.NoError(t, err)
item := items[0]
assert.Equal(t, int32(10), item.SeriesCount.Int32)
assert.Equal(t, int32(2), item.Volume.Int32)
assert.Equal(t, "DC Black Label", item.Imprint.String)
assert.Equal(t, "Mature", item.AgeRating.String)
assert.Equal(t, "https://example.com/comic", item.WebUrl.String)
assert.Equal(t, 9.2, item.CommunityRating.Float64)
})
t.Run("Ebook with universal fields", func(t *testing.T) {
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
LibraryID: ebookLibraryID,
Title: "Ebook with Universal Metadata",
FilePath: "/test/book.epub",
SeriesCount: pgtype.Int4{Int32: 7, Valid: true},
Volume: pgtype.Int4{Int32: 1, Valid: true},
Imprint: pgtype.Text{String: "HarperCollins", Valid: true},
AgeRating: pgtype.Text{String: "Everyone", Valid: true},
WebURL: pgtype.Text{String: "https://example.com/book", Valid: true},
CommunityRating: pgtype.Float8{Float64: 4.5, Valid: true},
})
require.NoError(t, err)
items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
LibraryID: ebookLibraryID,
})
require.NoError(t, err)
item := items[0]
assert.Equal(t, int32(7), item.SeriesCount.Int32)
assert.Equal(t, int32(1), item.Volume.Int32)
assert.Equal(t, "HarperCollins", item.Imprint.String)
assert.Equal(t, "Everyone", item.AgeRating.String)
assert.Equal(t, "https://example.com/book", item.WebUrl.String)
assert.Equal(t, 4.5, item.CommunityRating.Float64)
})
}
// TestComicSpecificFields tests comic-specific fields
func TestComicSpecificFields(t *testing.T) {
setup := setupDeviceTest(t)
defer setup.Server.Close()
ctx := context.Background()
libraryID := setup.CreateLibrary(t, "Comic Library", "comic")
t.Run("Alternate series info as JSONB", func(t *testing.T) {
alternateInfo := `{"alternate_series":"Ultimate X-Men","alternate_number":1,"alternate_count":12}`
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
LibraryID: libraryID,
Title: "X-Men with Alternate Series",
FilePath: "/test/xmen.cbz",
AlternateInfo: []byte(alternateInfo),
ScanInformation: pgtype.Text{String: "Scanned by Minutemen", Valid: true},
Summary: pgtype.Text{String: "Professor X creates mutant team", Valid: true},
MetadataNotes: pgtype.Text{String: "From collection", Valid: true},
})
require.NoError(t, err)
items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
LibraryID: libraryID,
})
require.NoError(t, err)
item := items[0]
assert.NotNil(t, item.AlternateInfo)
assert.JSONEq(t, alternateInfo, string(item.AlternateInfo))
assert.Equal(t, "Scanned by Minutemen", item.ScanInformation.String)
assert.Equal(t, "Professor X creates mutant team", item.Summary.String)
assert.Equal(t, "From collection", item.MetadataNotes.String)
})
}
```
**Important:** Use `setupDeviceTest(t)` from `cmd/server/tests/test_helpers_test.go`
- `setupDeviceTest` returns `*TestDeviceSetup` which has the `CreateLibrary()` helper method
- Internally calls `setupTestServer()` to set up the database and server
- Call `setupDeviceTest(t)` ONCE per test function, not per subtest
- The device setup provides user, auth tokens, and library creation helpers (you don't need to use the device-specific features)
- Verify `CommunityRating` is `pgtype.Float8` (DOUBLE PRECISION) in all tests
---
## Phase 7: Bruno API Tests
### 7.1 Create Bruno YAML Files
**Directory:** `bruno/media-items/`
**Files to Create:**
#### `bruno/media-items/search-with-reading-direction.yml`
```yaml
meta:
name: Search media items with reading direction filter
type: http
seq: 1
config:
test:
filter: search-reading-direction
workflow: search
http:
method: GET
url: "{{baseUrl}}/api/media-items?library_id={{libraryId}}&reading_direction=rtl"
headers:
Authorization: "Bearer {{accessToken}}"
```
#### `bruno/media-items/create-comic-with-metadata.yml`
```yaml
meta:
name: Upload comic with ComicInfo.xml
type: http
seq: 2
config:
test:
filter: comic-upload
workflow: upload
http:
method: POST
url: "{{baseUrl}}/api/media-items/upload"
headers:
Authorization: "Bearer {{accessToken}}"
Content-Type: "multipart/form-data"
body:
form_data:
library_id: "{{libraryId}}"
file:
type: file
src: test-files/manga-rtl.cbz
```
---
## Phase 8: Documentation
### 8.1 Update User Documentation
**File:** `docs/user/manga-reading-direction.md`
**Create New File:**
```markdown
# Manga Reading Direction Support
Bookhoard automatically detects and stores the reading direction for manga and comics.
## Supported Reading Directions
- **LTR (Left-to-Right)**: Western comics, manhwa, some manga
- **RTL (Right-to-Left)**: Traditional Japanese manga
- **Vertical**: Webtoons, manhwa (Korean webcomics)
- **Auto**: System will determine direction automatically
## How Reading Direction is Detected
1. **ComicInfo.xml**: If your comic archive contains `ComicInfo.xml` with the `Manga` field set to `YesAndRightToLeft`, Bookhoard will mark it as RTL
2. **Language**: Japanese comics without metadata are assumed to be RTL
3. **Genre**: Comics tagged as "webtoon" or "manhwa" are marked as vertical
4. **Manual**: You can manually override the reading direction in the metadata editor
## ComicInfo.xml Support
Bookhoard supports the industry-standard `ComicInfo.xml` format for comic metadata. When importing comics:
- If `ComicInfo.xml` exists inside the archive, its metadata is extracted
- The `Manga` field is used to set reading direction
- If a Calibre `metadata.opf` file exists in the same folder, both files are merged:
- Calibre metadata takes priority for title, author, publisher, etc.
- ComicInfo.xml takes priority for reading direction (since Calibre doesn't support it)
## Metadata Editor
You can view and edit reading direction in the metadata editor:
1. Navigate to a manga or comic in your library
2. Click "Edit Metadata"
3. Change the "Reading Direction" field
4. Save to update the database (and optionally write back to the file)
## Device Sync
Reading direction is synced to your devices:
- **KOReader**: Reading direction is set in the document metadata
- **Kobo**: Reading direction is stored in the Kobo metadata
- **Web Reader**: The reader automatically adjusts page turn direction based on reading direction
## Troubleshooting
### My manga is showing the wrong reading direction
1. Check if the comic has `ComicInfo.xml` inside the archive
2. Open the archive and verify the `Manga` field value:
- `YesAndRightToLeft` → RTL
- `Yes` or `No` → LTR
3. If no metadata exists, manually edit the reading direction in Bookhoard
### How do I add ComicInfo.xml to my comics?
Use a comic metadata editor such as:
- **ComicTagger** (cross-platform)
- **ComicRack** (Windows)
- **Komga** (can generate metadata for your library)
### My Calibre library has metadata.opf files. Will they work?
Yes! Bookhoard intelligently merges Calibre `metadata.opf` and `ComicInfo.xml`:
- Calibre metadata is used for title, author, series, etc.
- ComicInfo.xml is used for reading direction
- You get the best of both worlds
```
### 8.2 Update API Documentation
**File:** `docs/developer/api/media-items/search.md`
**Update:** Add `reading_direction` parameter to query parameters section
**Add:**
```
### Query Parameters
| Parameter | Type | Description | Example |
|-----------|------|-------------|---------|
| library_id | string | Filter by library UUID | `?library_id=uuid` |
| reading_direction | string | Filter by reading direction: `auto`, `ltr`, `rtl`, `vertical` | `?reading_direction=rtl` |
| manga_type | string | Filter by raw manga type: `unknown`, `no`, `yes`, `yes_and_right_to_left` | `?manga_type=yes_and_right_to_left` |
```
**File:** `docs/developer/api/media-items/object.md`
**Update:** Add `manga_type` and `reading_direction` to MediaItemSummary object
**Add:**
```
### MediaItemSummary Object
| Field | Type | Description |
|-------|------|-------------|
| ... existing fields ... |
| manga_type | string | Raw `Manga` field from ComicInfo.xml: `unknown`, `no`, `yes`, `yes_and_right_to_left` |
| reading_direction | string | Computed reading direction: `auto`, `ltr` (left-to-right), `rtl` (right-to-left), `vertical` (webtoons) |
```
---
## Phase 9: Build & Verification
### 9.1 Build Verification
**Commands:**
```bash
# Build backend
go build ./...
# Run tests
go test ./... -v
# Verify guidelines compliance
bash scripts/verify-guidelines.sh
```
### 9.2 Database Update
**Option 1: Recreate Database (Recommended - Loses Data)**
```bash
podman compose down -v # Delete all volumes
podman compose up -d # Start with fresh schema
```
**Option 2: Manual Migration (Preserves Data)**
```bash
podman exec bookhoard_db psql -U postgres -d bookhoard -c "
-- Add manga and comic metadata fields (14 new columns)
ALTER TABLE media_items
ADD COLUMN manga_type VARCHAR(30) DEFAULT 'unknown'
CHECK (manga_type IN ('unknown', 'no', 'yes', 'yes_and_right_to_left'));
ALTER TABLE media_items
ADD COLUMN reading_direction VARCHAR(20) DEFAULT 'auto'
CHECK (reading_direction IN ('auto', 'ltr', 'rtl', 'vertical'));
ALTER TABLE media_items
ADD COLUMN series_count INTEGER;
ALTER TABLE media_items
ADD COLUMN volume INTEGER;
ALTER TABLE media_items
ADD COLUMN imprint VARCHAR(255);
ALTER TABLE media_items
ADD COLUMN story_arc VARCHAR(255);
ALTER TABLE media_items
ADD COLUMN age_rating VARCHAR(20);
ALTER TABLE media_items
ADD COLUMN web_url VARCHAR(500);
ALTER TABLE media_items
ADD COLUMN metadata_notes TEXT;
ALTER TABLE media_items
ADD COLUMN is_black_and_white BOOLEAN DEFAULT FALSE;
ALTER TABLE media_items
ADD COLUMN community_rating DOUBLE PRECISION;
ALTER TABLE media_items
ADD COLUMN alternate_info JSONB;
ALTER TABLE media_items
ADD COLUMN scan_information TEXT;
ALTER TABLE media_items
ADD COLUMN summary TEXT;
-- Create indexes for efficient querying
CREATE INDEX IF NOT EXISTS idx_media_items_reading_direction
ON media_items(reading_direction)
WHERE reading_direction IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_media_items_story_arc
ON media_items(story_arc)
WHERE story_arc IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_media_items_imprint
ON media_items(imprint)
WHERE imprint IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_media_items_age_rating
ON media_items(age_rating)
WHERE age_rating IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_media_items_manga_type
ON media_items(manga_type)
WHERE manga_type IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_media_items_series_count
ON media_items(series_count)
WHERE series_count IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_media_items_volume
ON media_items(volume)
WHERE volume IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_media_items_alternate_info_gin
ON media_items USING GIN (alternate_info)
WHERE alternate_info IS NOT NULL;
-- Add column comments
COMMENT ON COLUMN media_items.manga_type IS 'Raw Manga field from ComicInfo.xml: unknown, no, yes, yes_and_right_to_left';
COMMENT ON COLUMN media_items.reading_direction IS 'Computed reading direction: auto, ltr, rtl, vertical';
COMMENT ON COLUMN media_items.series_count IS 'Total items in series (from ComicInfo.xml Count field or book series count)';
COMMENT ON COLUMN media_items.volume IS 'Volume/omnibus number for collected editions';
COMMENT ON COLUMN media_items.imprint IS 'Publisher imprint/subdivision (e.g., Vertigo, Icon, DC Black Label)';
COMMENT ON COLUMN media_items.story_arc IS 'Story arc name for grouping related issues (e.g., "The Dark Phoenix Saga", "Civil War")';
COMMENT ON COLUMN media_items.age_rating IS 'Age rating: Everyone, Teen, Mature, Adult';
COMMENT ON COLUMN media_items.web_url IS 'URL to info page (Goodreads, ComicVine, MangaUpdates, Audible, etc.)';
COMMENT ON COLUMN media_items.metadata_notes IS 'Notes from metadata files (ComicInfo.xml, EPUB, PDF) - distinct from user notes';
COMMENT ON COLUMN media_items.is_black_and_white IS 'Black and white flag (mostly comics)';
COMMENT ON COLUMN media_items.community_rating IS 'Community rating from metadata (scale 0.0-10.0) - distinct from user ratings';
COMMENT ON COLUMN media_items.alternate_info IS 'Alternate series information as JSONB: {alternate_series, alternate_number, alternate_count}';
COMMENT ON COLUMN media_items.scan_information IS 'Scan information (scanner group, resolution, etc.)';
COMMENT ON COLUMN media_items.summary IS 'Summary from ComicInfo.xml (may be merged with description from Calibre)';
"
```
**Option 3: Using SQL File**
```bash
# Save migration to file
cat > /tmp/add_comic_metadata.sql << 'EOF'
-- Add comprehensive metadata support (15 new columns)
-- Universal fields (apply to ebooks, audiobooks, comics)
ALTER TABLE media_items
ADD COLUMN manga_type VARCHAR(30) DEFAULT 'unknown'
CHECK (manga_type IN ('unknown', 'no', 'yes', 'yes_and_right_to_left'));
ALTER TABLE media_items
ADD COLUMN reading_direction VARCHAR(20) DEFAULT 'auto'
CHECK (reading_direction IN ('auto', 'ltr', 'rtl', 'vertical'));
ALTER TABLE media_items ADD COLUMN series_count INTEGER;
ALTER TABLE media_items ADD COLUMN volume INTEGER;
ALTER TABLE media_items ADD COLUMN imprint VARCHAR(255);
ALTER TABLE media_items ADD COLUMN age_rating VARCHAR(20);
ALTER TABLE media_items ADD COLUMN web_url VARCHAR(500);
ALTER TABLE media_items ADD COLUMN metadata_notes TEXT;
ALTER TABLE media_items ADD COLUMN community_rating DECIMAL(3,1);
-- Comic-specific fields
ALTER TABLE media_items ADD COLUMN story_arc VARCHAR(255);
ALTER TABLE media_items ADD COLUMN is_black_and_white BOOLEAN DEFAULT FALSE;
ALTER TABLE media_items ADD COLUMN alternate_info JSONB;
ALTER TABLE media_items ADD COLUMN scan_information TEXT;
ALTER TABLE media_items ADD COLUMN summary TEXT;
-- Create indexes
CREATE INDEX IF NOT EXISTS idx_media_items_reading_direction
ON media_items(reading_direction) WHERE reading_direction IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_media_items_story_arc
ON media_items(story_arc) WHERE story_arc IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_media_items_imprint
ON media_items(imprint) WHERE imprint IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_media_items_age_rating
ON media_items(age_rating) WHERE age_rating IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_media_items_manga_type
ON media_items(manga_type) WHERE manga_type IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_media_items_series_count
ON media_items(series_count) WHERE series_count IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_media_items_volume
ON media_items(volume) WHERE volume IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_media_items_alternate_info_gin
ON media_items USING GIN (alternate_info) WHERE alternate_info IS NOT NULL;
-- Add comments
COMMENT ON COLUMN media_items.manga_type IS 'Raw Manga field from ComicInfo.xml: unknown, no, yes, yes_and_right_to_left';
COMMENT ON COLUMN media_items.reading_direction IS 'Computed reading direction: auto, ltr, rtl, vertical';
COMMENT ON COLUMN media_items.series_count IS 'Total items in series';
COMMENT ON COLUMN media_items.volume IS 'Volume/omnibus number for collected editions';
COMMENT ON COLUMN media_items.imprint IS 'Publisher imprint (e.g., Vertigo, Icon)';
COMMENT ON COLUMN media_items.age_rating IS 'Age rating: Everyone, Teen, Mature, Adult';
COMMENT ON COLUMN media_items.web_url IS 'URL to info page (Goodreads, ComicVine, MangaUpdates, Audible, etc.)';
COMMENT ON COLUMN media_items.metadata_notes IS 'Notes from metadata files (not user notes)';
COMMENT ON COLUMN media_items.community_rating IS 'Community rating (0.0-10.0)';
COMMENT ON COLUMN media_items.story_arc IS 'Story arc name for grouping';
COMMENT ON COLUMN media_items.is_black_and_white IS 'Black and white flag (mostly comics)';
COMMENT ON COLUMN media_items.alternate_info IS 'Alternate series as JSONB';
COMMENT ON COLUMN media_items.scan_information IS 'Scan information';
COMMENT ON COLUMN media_items.summary IS 'Summary from ComicInfo.xml';
EOF
# Apply migration
podman exec -i bookhoard_db psql -U postgres -d bookhoard < /tmp/add_comic_metadata.sql
```
---
## Phase 10: Git Commit Structure
### Commit Strategy Overview
**Principle**: Make good organized git commits with detailed messages for the entire project (not just what you changed). Run git add, commit, push sequentially as separate commands. Do not make one giant commit unless you are sure all files changed are for the same related edit.
**Total Commits**: 10 separate commits (organized by logical grouping)
### Commits (Sequential, No `&&` Chaining)
**IMPORTANT**: Run these commands SEQUENTIALLY, waiting for each to complete before starting the next.
```bash
# Commit 1: Database schema changes
git add database/schema/schema.sql
git commit -m "feat: add comprehensive comic metadata support to media_items table
Add 16 new columns to support all ComicInfo.xml v2.0 fields:
- manga_type: Raw Manga field (unknown, no, yes, yes_and_right_to_left)
- reading_direction: Computed direction (auto, ltr, rtl, vertical)
- series_count: Total issues in series
- volume: Volume number for collected editions
- imprint: Publisher imprint (e.g., Vertigo, Icon)
- story_arc: Story arc name for grouping issues
- age_rating: Age rating (Everyone, Teen, Mature, Adult)
- genre_comic: Comic-specific genre
- web_url: URL to comic info page
- metadata_notes: Notes from ComicInfo.xml (distinct from user notes)
- is_black_and_white: B/W flag
- community_rating: Community rating (0-10 scale)
- alternate_info: JSONB for alternate series data
- scan_information: Scan info (scanner, resolution)
- summary: Summary from ComicInfo.xml
Add 8 indexes for efficient comic and universal metadata queries:
- idx_media_items_reading_direction
- idx_media_items_story_arc
- idx_media_items_imprint
- idx_media_items_age_rating
- idx_media_items_manga_type
- idx_media_items_series_count (NEW)
- idx_media_items_volume (NEW)
- idx_media_items_alternate_info_gin
Supports all 19 ComicInfo.xml fields plus universal fields for all formats:
Comic-specific: reading direction, story arc, scan information, alternate info, B/W flag
Universal (ebooks, audiobooks, comics): age rating, series count, volume, imprint, web URL, metadata notes, community rating
Genre/tag processing: ALL genres appear in tags array without duplication
- Uses existing `genre` column for primary genre (first genre tag)
- Populates `tags` array with all genres from metadata files
- Works for EPUB, ComicInfo.xml, PDF metadata
- No genre_comic column needed (reuses existing genre column)"
git add internal/database/
git commit -m "chore: regenerate database models for comprehensive metadata support
- Regenerate models.go with sqlc after schema changes
- Update CreateMediaItemParams with 15 new fields (was 16, removed genre_comic)
- Update all queries that return MediaItems to include new columns
- Support pgtype types for TEXT, JSONB, DECIMAL, BOOLEAN fields"
git add internal/services/media_scanner.go
git commit -m "feat: add smart metadata merging with genre/tag processing
- Add all 19 fields to ComicInfo struct (complete v2.0 support)
- Add universal + comic fields to MediaMetadata struct (15 fields total)
- Implement comprehensive mergeMetadata() for intelligent merging:
- Priority: Calibre metadata.opf → ComicInfo.xml → folder structure → filename
- Always extract comic fields even when metadata.opf exists
- Merge Characters, Teams, Locations into tags array via extractGenreTagsFromComicInfo()
- Build alternate_info JSONB from AlternateSeries/Number/Count
- Add genre/tag processing functions (universal for ALL formats):
- processGenresAndTags(): Ensures ALL genres appear in tags without duplication
- extractGenreTagsFromEPUB(): Extract genres from EPUB <dc:subject> tags
- extractGenreTagsFromComicInfo(): Extract Genre + Tags + Characters + Teams + Locations
- Uses existing `genre` column for primary genre (first genre tag)
- Populates `tags` array with all genres (no duplicates via containsTag())
- Add helper functions:
- normalizeMangaType(): Standardize Manga field values
- determineReadingDirection(): Compute from Manga + language + genre
- normalizeAgeRating(): Standardize age rating values
- containsTag(): Prevent duplicate tags
- Update extractMetadata() to use smart merging
- Remove duplicate comic metadata extraction in processMediaFile()
- Calibre metadata.opf takes priority for basic fields
- ComicInfo.xml fills gaps and provides comic-specific fields
- Ensures all 19 ComicInfo.xml fields + 5 universal fields are extracted
Supports complete ComicInfo.xml v2.0 field extraction:
- Series organization (Count, Volume, AlternateSeries)
- Publisher hierarchy (Publisher, Imprint)
- Content classification (Genre, Manga, AgeRating, StoryArc, SeriesGroup)
- Narrative elements (Summary, Notes, Characters, Teams, Locations)
- Publication metadata (Year, Month, Day, Web, PageCount, LanguageISO)
- Credits (Writer, Penciller, Inker, Colorist, Letterer, CoverArtist, Editor)
- Format (BlackAndWhite, Manga)
- Community data (CommunityRating, MainCharacterOrTeam, Review)
- Technical (ScanInformation)
Universal metadata fields (apply to ebooks, audiobooks, comics):
- age_rating: Age-based content classification
- series_count: Total items in series
- volume: Collected edition/omnibus number
- imprint: Publisher subdivision
- web_url: Info page URL (Goodreads, Audible, ComicVine, etc.)
- metadata_notes: Notes from metadata files (not user notes)
- community_rating: Pre-existing ratings from metadata"
git add internal/services/media_scanner_comic_test.go
git commit -m "test: add unit tests for metadata extraction and genre/tag processing
- Add TestNormalizeMangaType with all enum values and edge cases
- Add TestDetermineReadingDirection with Manga, language, genre heuristics
- Add TestNormalizeAgeRating with all rating values
- Add TestProcessGenresAndTags for universal genre/tag logic
- Add TestExtractGenreTagsFromEPUB for EPUB subject processing
- Add TestExtractGenreTagsFromComicInfo for comic tag processing
- Add TestMergeMetadata for metadata.opf + ComicInfo.xml merge scenarios
- Test all 19 ComicInfo.xml fields + 5 universal fields are extracted correctly
- Test alternate_info JSONB serialization
- Test tag deduplication (no duplicate genres in tags array)
- Test genre → primary genre, all genres → tags logic
- Cover RTL, LTR, vertical reading direction detection
- Test webtoon, manhwa, Japanese manga detection
- Test age rating normalization (Everyone, Teen, Mature, Adult)
- Test community rating extraction
- Test series count and volume extraction"
git add cmd/server/tests/comic_metadata_test.go
git commit -m "test: add integration tests for comic metadata extraction
- Add TestComicMetadataExtraction for CBZ with/without ComicInfo.xml
- Test all 19 fields are stored correctly in database
- Add TestCalibreComicMerge for metadata.opf + ComicInfo.xml merging
- Verify priority: metadata.opf → ComicInfo.xml → folder structure
- Add TestReadingDirectionAPI for API response validation
- Test alternate_info JSONB in API responses
- Test search by story_arc, imprint, age_rating
- Test series_count and volume in API responses
- Test Characters/Teams/Locations merged into tags
- Use setupTestServer() helper from test_helpers
- Test no user, user, and admin contexts"
git add web/src/types/api.d.ts
git commit -m "feat: add all comic metadata fields to MediaItemSummary type
- Add manga_type field with enum values
- Add reading_direction field with ltr/rtl/vertical/auto options
- Add series_count, volume, imprint, story_arc
- Add age_rating, genre_comic, web_url
- Add metadata_notes (distinct from user notes)
- Add is_black_and_white, community_rating
- Add alternate_info JSONB object
- Add scan_information, summary
- Ensure TypeScript types match Go database models
- Support all 15 new metadata fields (5 universal + 10 comic-specific)"
git add bruno/media-items/
git commit -m "test: add Bruno API tests for comic metadata feature
- Add search-with-reading-direction.yml for filtering by direction
- Add search-by-story-arc.yml for story arc queries
- Add search-by-imprint.yml for imprint filtering
- Add search-by-age-rating.yml for age rating queries (universal)
- Add search-by-series-count.yml for series filtering (universal)
- Add create-comic-with-metadata.yml for uploading with ComicInfo.xml
- Test API contracts for all new fields (universal + comic-specific)
- Verify JSONB serialization of alternate_info
- Test genre/tag processing (all genres appear in tags without duplication)"
git add docs/user/manga-reading-direction.md
git commit -m "docs: add comprehensive user documentation for metadata features
- Explain all 19 ComicInfo.xml fields
- Document reading direction detection (Manga + language + genre)
- Document smart merging with Calibre metadata.opf
- Explain universal fields (age_rating, series_count, volume, imprint, web_url)
- Explain comic-specific fields (story arc, scan_information, alternate_info)
- Document genre/tag processing (all genres appear in tags)
- Add troubleshooting guide for missing metadata
- Document metadata editor usage
- Explain how to add ComicInfo.xml to comics
- List recommended metadata editor tools
- Document device sync for reading direction"
git add docs/developer/api/media-items/
git commit -m "docs: update API documentation for all metadata fields
- Add reading_direction, manga_type query parameters
- Add story_arc, imprint, age_rating filters
- Add series_count, volume, web_url filters
- Update MediaItemSummary object with all 15 new fields
- Document alternate_info JSONB structure
- Document genre/tag processing behavior
- Document enum values for manga_type and reading_direction
- Document age_rating enum values
- Add examples for filtering by metadata
- Note universal fields apply to ebooks, audiobooks, comics"
git push
```
---
## Summary of Changes
### Files Modified
1. `database/schema/schema.sql` - Add 15 new columns, 8 indexes, column comments
2. `internal/database/models.go` - Regenerated by sqlc with new fields
3. `internal/services/media_scanner.go` - Smart merging logic + genre/tag processing
4. `internal/services/media_scanner_comic_test.go` - Unit tests for all new fields
5. `web/src/types/api.d.ts` - TypeScript types with all metadata
6. `docs/user/manga-reading-direction.md` - User docs (new file)
7. `docs/developer/api/media-items/*.md` - API docs updates
### Files Created
1. `cmd/server/tests/comic_metadata_test.go` - Integration tests for all fields
2. `bruno/media-items/search-with-reading-direction.yml` - API test
3. `bruno/media-items/create-comic-with-metadata.yml` - API test
4. `bruno/media-items/search-by-story-arc.yml` - API test (new)
5. `bruno/media-items/search-by-imprint.yml` - API test (new)
6. `bruno/media-items/search-by-age-rating.yml` - API test (new)
7. `bruno/media-items/search-by-series-count.yml` - API test (new)
### Key Features
**Complete ComicInfo.xml Support**: Extracts ALL 19 fields from ComicInfo.xml v2.0
**Universal Metadata Fields**: 5 fields apply to ALL formats (ebooks, audiobooks, comics):
- `age_rating`: Age-based content classification
- `series_count`: Total items in series
- `volume`: Collected edition/omnibus number
- `imprint`: Publisher subdivision
- `web_url`: Info page URL (Goodreads, Audible, ComicVine, etc.)
**Comic-Specific Fields**: 10 fields for comics:
- Reading direction (`manga_type`, `reading_direction`)
- Story arc (`story_arc`)
- Scan info (`scan_information`)
- Alternate series (`alternate_info` JSONB)
- Format (`is_black_and_white`)
- Metadata (`summary`, `metadata_notes`, `community_rating`)
**Smart Metadata Merging**: Calibre `metadata.opf` + `ComicInfo.xml` + folder structure
**Reading Direction Detection**: From `Manga` field + language heuristics + genre tags
**Genre/Tag Processing**: ALL genres appear in tags array without duplication
- Uses existing `genre` column for primary genre (first genre tag)
- Processes EPUB `<dc:subject>` tags
- Processes ComicInfo `Genre` + `Tags` + `Characters` + `Teams` + `Locations`
- Deduplication via `containsTag()` helper
**Database Storage**: Raw `manga_type` + computed `reading_direction` + 13 other fields
**Full Stack**: Database → Go → API → TypeScript → Frontend
**Testing**: Unit tests, integration tests, Bruno API tests
**Documentation**: User guide, API reference, troubleshooting
### All 19 ComicInfo.xml Fields Extracted
| Field | Database Column | Type | Description |
| ------------------- | ------------------ | ------------ | ------------------------------------------------------------------------ |
| Title | title | VARCHAR(255) | Already exists |
| Series | series | VARCHAR(255) | Already exists |
| Number | series_number | INTEGER | Already exists |
| Count | series_count | INTEGER | **NEW**: Total issues in series (UNIVERSAL) |
| Volume | volume | INTEGER | **NEW**: Volume number (UNIVERSAL) |
| AlternateSeries | alternate_info | JSONB | **NEW**: Alternate series info (comic-specific) |
| AlternateNumber | alternate_info | JSONB | **NEW**: Alternate number (comic-specific) |
| AlternateCount | alternate_info | JSONB | **NEW**: Alternate count (comic-specific) |
| Summary | summary | TEXT | **NEW**: Comic summary (can merge with description) |
| Notes | metadata_notes | TEXT | **NEW**: Notes from metadata files (UNIVERSAL, distinct from user notes) |
| Year | copyright_year | INTEGER | Already exists |
| Month | (derived) | - | Stored in date_published |
| Day | (derived) | - | Stored in date_published |
| Writer | author | VARCHAR(255) | Already exists |
| Penciller | contributors | TEXT[] | Merged into existing |
| Inker | contributors | TEXT[] | Merged into existing |
| Colorist | contributors | TEXT[] | Merged into existing |
| Letterer | contributors | TEXT[] | Merged into existing |
| CoverArtist | contributors | TEXT[] | Merged into existing |
| Publisher | publisher | VARCHAR(255) | Already exists |
| Imprint | imprint | VARCHAR(255) | **NEW**: Publisher imprint (UNIVERSAL) |
| Genre | genre | VARCHAR(100) | Already exists + enhanced via processGenresAndTags() |
| Tags | tags | TEXT[] | Already exists + enhanced with Characters/Teams/Locations |
| Web | web_url | VARCHAR(500) | **NEW**: Info page URL (UNIVERSAL) |
| PageCount | page_count | INTEGER | Already exists |
| LanguageISO | language | VARCHAR(10) | Already exists |
| Manga | manga_type | VARCHAR(30) | **NEW**: Raw manga field (comic-specific) |
| **Derived** | reading_direction | VARCHAR(20) | **NEW**: Computed from Manga field (comic-specific) |
| BlackAndWhite | is_black_and_white | BOOLEAN | **NEW**: B/W flag (mostly comic-specific) |
| StoryArc | story_arc | VARCHAR(255) | **NEW**: Story arc name (comic-specific) |
| SeriesGroup | (future) | - | Not stored (can add later) |
| AgeRating | age_rating | VARCHAR(20) | **NEW**: Age rating (UNIVERSAL) |
| CommunityRating | community_rating | DOUBLE PRECISION | **NEW**: 0-10 rating (UNIVERSAL, from metadata files) |
| MainCharacterOrTeam | tags | TEXT[] | Merged into tags via processGenresAndTags() |
| Review | (future) | - | Not stored (user notes exist) |
| ScanInformation | scan_information | TEXT | **NEW**: Scan info (comic-specific) |
| Characters | tags | TEXT[] | Merged into tags via processGenresAndTags() |
| Teams | tags | TEXT[] | Merged into tags via processGenresAndTags() |
| Locations | tags | TEXT[] | Merged into tags via processGenresAndTags() |
### Universal Fields (Apply to Ebooks, Audiobooks, Comics)
| Field | Database Column | Formats | Examples |
| ------------------- | ------------------ | ------------ | ------------------------------------------------------------------------------------ |
| age_rating | age_rating | All | Everyone, Teen, Mature, Adult |
| series_count | series_count | All | "7" (Harry Potter), "12" (One Piece) |
| volume | volume | All | "1" (Omnibus), "2" (Collected Edition) |
| imprint | imprint | All | "Vertigo" (comics), "HarperCollinsEpic" (books) |
| web_url | web_url | All | Goodreads, Audible, ComicVine, MangaUpdates URLs |
| metadata_notes | metadata_notes | All | Publisher notes, scan info, EPUB annotations |
| community_rating | community_rating | All | Pre-existing ratings from metadata sources |
| Notes | metadata_notes | TEXT | **NEW**: Metadata notes (not user notes) |
| Year | copyright_year | INTEGER | Already exists |
| Month | (derived) | - | Stored in date_published |
| Day | (derived) | - | Stored in date_published |
| Writer | author | VARCHAR(255) | Already exists |
| Penciller | contributors | TEXT[] | Merged into existing |
| Inker | contributors | TEXT[] | Merged into existing |
| Colorist | contributors | TEXT[] | Merged into existing |
| Letterer | contributors | TEXT[] | Merged into existing |
| CoverArtist | contributors | TEXT[] | Merged into existing |
| Publisher | publisher | VARCHAR(255) | Already exists |
| Imprint | imprint | VARCHAR(255) | **NEW**: Publisher imprint (UNIVERSAL) |
| Genre | genre | VARCHAR(100) | Already exists + enhanced via processGenresAndTags() |
| Tags | tags | TEXT[] | Already exists + enhanced with Characters/Teams/Locations via processGenresAndTags() |
| Web | web_url | VARCHAR(500) | **NEW**: Info page URL (UNIVERSAL) |
| PageCount | page_count | INTEGER | Already exists |
| LanguageISO | language | VARCHAR(10) | Already exists |
| Manga | manga_type | VARCHAR(30) | **NEW**: Raw manga field |
| **Derived** | reading_direction | VARCHAR(20) | **NEW**: Computed from Manga field |
| BlackAndWhite | is_black_and_white | BOOLEAN | **NEW**: B/W flag |
| StoryArc | story_arc | VARCHAR(255) | **NEW**: Story arc name |
| SeriesGroup | (future) | - | Not stored (can add later) |
| AgeRating | age_rating | VARCHAR(20) | **NEW**: Age rating |
| CommunityRating | community_rating | DOUBLE PRECISION | **NEW**: 0-10 rating |
| MainCharacterOrTeam | tags | TEXT[] | Merged into tags |
| Review | (future) | - | Not stored (user notes exist) |
| ScanInformation | scan_information | TEXT | **NEW**: Scan info |
| Characters | tags | TEXT[] | Merged into tags |
| Teams | tags | TEXT[] | Merged into tags |
| Locations | tags | TEXT[] | Merged into tags |
---
## Testing Checklist
Before considering this feature complete:
### Schema & Database
- [ ] Database schema updated with 14 new columns (community_rating is DOUBLE PRECISION, not DECIMAL)
- [ ] All 8 indexes created successfully (already in schema)
- [ ] Column comments added for all new fields
- [ ] Database models regenerated with sqlc after schema change (CommunityRating becomes pgtype.Float8)
- [ ] Local database updated (Option 1: recreate database, or Option 2: manual migration)
### Code Changes
- [ ] ComicInfo struct updated with all 19 fields
- [ ] MediaMetadata struct updated with 15 fields (5 universal + 10 comic-specific)
- [ ] mergeMetadata() function implements smart merging
- [ ] Genre/tag processing functions implemented:
- [ ] processGenresAndTags() for universal genre/tag logic
- [ ] extractGenreTagsFromEPUB() for EPUB subject tags
- [ ] extractGenreTagsFromComicInfo() for comic tags
- [ ] normalizeMangaType() function handles all enum values
- [ ] determineReadingDirection() implements heuristics
- [ ] normalizeAgeRating() function standardizes ratings
- [ ] containsTag() helper prevents duplicate tags
- [ ] processMediaFile() updated with all new fields
- [ ] TypeScript types updated with all metadata fields
### Testing
- [ ] Unit tests pass (`go test ./internal/services/...`)
- [ ] Integration tests pass (`go test ./cmd/server/tests/...`)
- [ ] Bruno API tests pass
- [ ] Test coverage for all 19 ComicInfo.xml fields
### Manual Testing - Core Functionality
- [ ] Upload CBZ with `ComicInfo.xml` (Manga=YesAndRightToLeft)
- [ ] Verify RTL reading direction in database
- [ ] Upload CBZ with `metadata.opf` + `ComicInfo.xml`
- [ ] Verify smart merging: Calibre fields + ComicInfo.xml fields
- [ ] API responses include all new fields
- [ ] Search by `reading_direction` filter works
- [ ] Search by `story_arc` filter works
- [ ] Search by `imprint` filter works
- [ ] Search by `age_rating` filter works
- [ ] Search by `series_count` (e.g., "show complete series")
- [ ] Characters/Teams/Locations added to tags array
### Manual Testing - Specific Fields (Universal + Comic)
- [ ] `series_count` displays correctly in UI
- [ ] `volume` displays correctly in UI
- [ ] `imprint` displays and filters correctly (test with comics AND ebooks)
- [ ] `story_arc` groups related comics
- [ ] `age_rating` filters work (test with comics AND ebooks - parental controls)
- [ ] `web_url` links work (test with Goodreads, ComicVine, etc.)
- [ ] `metadata_notes` display (distinct from user notes)
- [ ] `community_rating` displays (distinct from user ratings in media_ratings table)
- [ ] `is_black_and_white` flag works
- [ ] `alternate_info` JSONB stores/retrieves correctly
- [ ] `scan_information` displays
- [ ] `summary` merges with description appropriately
### Manual Testing - Genre/Tag Processing
- [ ] EPUB subjects: first subject → genre, all subjects → tags
- [ ] ComicInfo Genre: → genre column
- [ ] ComicInfo Tags: → tags array
- [ ] ComicInfo Characters: → tags array
- [ ] ComicInfo Teams: → tags array
- [ ] ComicInfo Locations: → tags array
- [ ] No duplicate genres in tags (deduplication works)
- [ ] Primary genre set correctly (first genre tag wins)
### Documentation
- [ ] User documentation renders at `/docs` endpoint
- [ ] API documentation updated with all new fields
- [ ] Docs search finds new content
- [ ] Code examples in docs work
### Build & Verification
- [ ] `go build ./...` succeeds
- [ ] `bash scripts/verify-guidelines.sh` passes (0 errors)
- [ ] No critical functionality broken
- [ ] Git diff shows only intended changes
---
**Document Version:** 3.0
**Created:** March 29, 2026
**Updated:** March 29, 2026
**Changes:**
- v2.0: Added all 19 ComicInfo.xml fields (from 2 to 16 columns)
- v3.0: Made 5 fields universal (age_rating, series_count, volume, imprint, web_url)
- v3.0: Removed genre_comic column (use existing genre + processGenresAndTags)
- v3.0: Added genre/tag processing logic for ALL formats
- v3.0: Total: 15 new columns (5 universal + 10 comic-specific)
**Status:** Ready for Implementation