diff --git a/IMPLEMENTATION_PLAN_MERGE_METADATA_READING_DIRECTION.md b/IMPLEMENTATION_PLAN_MERGE_METADATA_READING_DIRECTION.md
index db3d5b0..485be90 100644
--- a/IMPLEMENTATION_PLAN_MERGE_METADATA_READING_DIRECTION.md
+++ b/IMPLEMENTATION_PLAN_MERGE_METADATA_READING_DIRECTION.md
@@ -1,9 +1,57 @@
+
+
# Implementation Plan: Smart Metadata Merge & Complete ComicInfo.xml Support
**Scope:** Extract ALL 19 ComicInfo.xml v2.0 fields (not just reading direction)
**Status:** Updated - Full ComicInfo.xml support (March 29, 2026)
+---
+
+## ⚠️ PLAN UPDATES (March 29, 2026)
+
+This implementation plan has been corrected with the following changes:
+
+1. **Context-based location markers**: All line number references have been replaced with 3-line context markers (before/after) to make locations easier to find even after code insertions shift line numbers.
+
+2. **Phase 1 (Database Schema)**: Marked as mostly complete. The schema columns, indexes, and comments are already in place. Only the SQL INSERT query needs updating.
+
+3. **Phase 4.1 (SQL Query)**: Added the complete SQL query code showing the 14 new columns and parameters.
+
+4. **Phase 4.4 (TODO Comment)**: Removed - this TODO comment doesn't exist in the current codebase.
+
+5. **Phase 2.2 (MediaMetadata)**: Updated with exact location of the struct (starts around line 42, not line 101).
+
+6. **Phase 3.4 (extractMetadata)**: Updated with context-based location and clarified that the fallback switch statement (`.epub`, `.pdf`) is removed as this logic moves into `mergeMetadata()`.
+
+7. **Phase 4.4 (Duplicate Code)**: Renumbered from 4.5 and added context-based location markers.
+
+8. **Fixed function name**: Corrected `processNewMediaItems()` → `processMediaFile()` throughout the plan. This is the actual function name in media_scanner.go.
+
+9. **Changed community_rating to DOUBLE PRECISION**: Changed from `DECIMAL(3,1)` to `DOUBLE PRECISION` for simpler code. This eliminates the awkward `pgtype.Numeric` conversion and uses simple `pgtype.Float8` instead. Matches ComicInfo.xml float64 type naturally with no string conversion needed. The floating-point precision error is negligible for 0-10 ratings (< 0.00001%).
+
+**What's Already Done:**
+
+- Database schema columns (lines 137-161 in schema.sql)
+- Database indexes (lines 424-461 in schema.sql)
+- Go models regenerated (MediaItems struct lines 229-256 in models.go)
+- CreateMediaItemParams struct (lines 586-599 in queries.sql.go)
+
+**What Still Needs Doing:**
+
+- Update CreateMediaItem SQL query in queries.sql
+- Update ComicInfo struct with new fields
+- Update MediaMetadata struct with new fields
+- Create mergeMetadata() function
+- Update extractMetadata() function
+- Update CreateMediaItem() call with new parameters
+- Remove duplicate comic extraction code
+- Update TypeScript types
+- Write tests
+- Write documentation
+
+---
+
## Overview
Add intelligent metadata merging for ALL media types (ebooks, comics, etc.) with comprehensive support for ALL ComicInfo.xml metadata fields (19 total fields). This feature will:
@@ -20,6 +68,7 @@ Add intelligent metadata merging for ALL media types (ebooks, comics, etc.) with
## Project Context
**Current State:**
+
- ✅ Parses Calibre `metadata.opf` sidecar files (priority)
- ✅ Parses `ComicInfo.xml` from comic archives (fallback only)
- ✅ Extracts basic comic metadata (title, series, number, publisher, writer)
@@ -40,7 +89,11 @@ Add intelligent metadata merging for ALL media types (ebooks, comics, etc.) with
**Location:** After line 133 (after `kobo_metadata JSONB`)
-**Changes:**
+**Status:** ✅ **Already completed** - All columns, indexes, and comments are in place
+
+**Note:** The schema has already been updated with all 14 new columns. The only change needed is to ensure `community_rating` is `DOUBLE PRECISION` (which it already is in the current schema).
+
+**Existing Schema (already in place):**
```sql
-- Manga and comic reading direction support
@@ -67,7 +120,7 @@ is_black_and_white BOOLEAN, -- Black and white flag (mostly comics, some i
-- Additional metadata (applies to all formats)
metadata_notes TEXT, -- Notes from metadata files (ComicInfo.xml, EPUB, PDF) - distinct from user notes
-community_rating DECIMAL(3,1), -- Pre-existing community rating from metadata (0.0-10.0) - distinct from user ratings
+community_rating DOUBLE PRECISION, -- Pre-existing community rating from metadata (0.0-10.0) - distinct from user ratings
-- Alternate series information (JSONB for flexible schema - comic-specific)
alternate_info JSONB, -- Stores AlternateSeries, AlternateNumber, AlternateCount
@@ -80,13 +133,22 @@ scan_information TEXT, -- Scan information (scanner group, resolution
summary TEXT, -- Summary from ComicInfo.xml (may be merged with description from Calibre)
```
+**Summary of Phase 1:**
+- ✅ All 14 columns already added to schema
+- ✅ All 8 indexes already created
+- ✅ All column comments already added
+- ✅ `community_rating` is `DOUBLE PRECISION` (not DECIMAL)
+- ⚠️ **Action needed:** Only if you haven't regenerated sqlc models yet
+
### 1.2 Add Indexes for Comic and Universal Metadata Queries
+**Note:** The indexes for these fields have already been created in the schema. This section is for reference only.
+
**File:** `database/schema/schema.sql`
-**Location:** After line 388 (after `idx_media_items_series`)
+**Location:** Find the section with existing comic metadata indexes (already implemented)
-**Changes:**
+**Existing Indexes (Already in schema.sql):**
```sql
-- Index for filtering by reading direction (for manga/comic libraries)
@@ -130,6 +192,8 @@ ON media_items USING GIN (alternate_info)
WHERE alternate_info IS NOT NULL;
```
+**Note:** These indexes are already in the schema and do not need to be added again.
+
### 1.3 Add Column Comments
**File:** `database/schema/schema.sql`
@@ -156,7 +220,7 @@ COMMENT ON COLUMN media_items.web_url IS 'URL to info page (Goodreads, ComicVine
-- Additional metadata (applies to all formats)
COMMENT ON COLUMN media_items.metadata_notes IS 'Notes from metadata files (ComicInfo.xml, EPUB, PDF) - distinct from user notes in media_notes table';
-COMMENT ON COLUMN media_items.community_rating IS 'Pre-existing community rating from metadata files (scale 0.0-10.0) - distinct from user ratings in media_ratings table';
+COMMENT ON COLUMN media_items.community_rating IS 'Pre-existing community rating from metadata files (scale 0.0-10.0, DOUBLE PRECISION) - distinct from user ratings in media_ratings table';
COMMENT ON COLUMN media_items.summary IS 'Summary from ComicInfo.xml (may be merged with description from Calibre)';
```
@@ -169,6 +233,7 @@ cd internal/database && sqlc generate
```
**Expected Changes:**
+
- `CreateMediaItemParams` struct will include `MangaType` and `ReadingDirection` fields
- `UpdateMediaItemIdentifiersParams` struct may need updating
- All queries that return `MediaItems` will include new columns
@@ -181,11 +246,17 @@ cd internal/database && sqlc generate
**File:** `internal/services/media_scanner.go`
-**Location:** Lines 1383-1403
+**Location:** Find the struct starting with:
+
+```go
+// ComicInfo represents metadata from ComicInfo.xml
+type ComicInfo struct {
+```
**Current Code:**
```go
+// ComicInfo represents metadata from ComicInfo.xml
type ComicInfo struct {
XMLName xml.Name `xml:"ComicInfo"`
Title string `xml:"Title"`
@@ -264,26 +335,57 @@ type ComicInfo struct {
**File:** `internal/services/media_scanner.go`
-**Location:** Around line 101 (in `CalibreOPFMetadata` or similar section)
-
-**Note:** Verify if `MediaMetadata` struct exists. If not, create it.
-
-**Add Fields:**
+**Location:** Find the struct starting with:
```go
+// MediaMetadata contains extracted metadata for media files (ebooks, comics, manga)
type MediaMetadata struct {
- // ... existing fields ...
- Title string
- Author string
- ISBN string
- Description string
- Publisher string
- Series string
- SeriesNumber int32
- Tags []string
- Contributors []string
- PublishDate time.Time
- ASIN string
+```
+
+**Current Code:**
+
+```go
+// MediaMetadata contains extracted metadata for media files (ebooks, comics, manga)
+type MediaMetadata struct {
+ Title string
+ Author string
+ Description string
+ Series string
+ SeriesNumber int32
+ Publisher string
+ PublishDate time.Time
+ Contributors []string
+ CoverPath string
+ ISBN string
+ ASIN string
+ Tags []string
+
+ FileHashInfo *HashInfo
+ FileFormats []*FormatInfo
+}
+```
+
+**Updated Code:**
+
+```go
+// MediaMetadata contains extracted metadata for media files (ebooks, comics, manga)
+type MediaMetadata struct {
+ // Existing fields
+ Title string
+ Author string
+ Description string
+ Series string
+ SeriesNumber int32
+ Publisher string
+ PublishDate time.Time
+ Contributors []string
+ CoverPath string
+ ISBN string
+ ASIN string
+ Tags []string
+
+ FileHashInfo *HashInfo
+ FileFormats []*FormatInfo
// NEW: Reading direction fields (from ComicInfo.xml or computed)
MangaType string // Raw ComicInfo.xml Manga field
@@ -298,7 +400,7 @@ type MediaMetadata struct {
AgeRating string // Age rating (Everyone, Teen, Mature, Adult)
WebURL string // URL to info page (Goodreads, ComicVine, etc.)
MetadataNotes string // Notes from metadata files (not user notes)
- CommunityRating float64 // Pre-existing community rating (0-10)
+ CommunityRating float64 // Pre-existing community rating (0.0-10.0) - maps to DOUBLE PRECISION in database
// Comic-specific fields
StoryArc string // Story arc name
@@ -317,7 +419,14 @@ type MediaMetadata struct {
**File:** `internal/services/media_scanner.go`
-**Location:** After `extractCalibreSidecar()` function (around line 750)
+**Location:** Insert this new function after `extractCalibreSidecar()` function. Find the function ending with:
+
+```go
+ return metadata
+}
+```
+
+and before the next function. The new `mergeMetadata()` function should be inserted here.
**New Function:**
@@ -661,14 +770,17 @@ if ext == ".cbz" || ext == ".cbr" || ext == ".cb7" || ext == ".cbt" {
**Examples:**
**EPUB:**
+
- Input: `FantasyAdventure`
- Result: `genre` = "Fantasy", `tags` = ["Fantasy", "Adventure"]
**ComicInfo.xml:**
+
- Input: `ActionAdventure, FightingNaruto, Sasuke`
- Result: `genre` = "Action", `tags` = ["Action", "Adventure", "Fighting", "Naruto", "Sasuke"]
**Calibre metadata.opf:**
+
- Input: Multiple `` tags
- Result: `genre` = first subject, `tags` = all subjects
@@ -676,9 +788,13 @@ if ext == ".cbz" || ext == ".cbr" || ext == ".cb7" || ext == ".cbt" {
**File:** `internal/services/media_scanner.go`
-**Location:** Lines 752-798
+**Location:** Find the function starting with:
-**Current Code (lines 752-764):**
+```go
+func (s *MediaScanner) extractMetadata(path string) (*MediaMetadata, error) {
+```
+
+**Current Code (first ~15 lines of the function):**
```go
func (s *MediaScanner) extractMetadata(path string) (*MediaMetadata, error) {
@@ -694,6 +810,10 @@ func (s *MediaScanner) extractMetadata(path string) (*MediaMetadata, error) {
return metadata, nil
}
+
+ // EXISTING: Fallback to embedded metadata
+ ext := strings.ToLower(filepath.Ext(path))
+ ...
```
**Updated Code:**
@@ -715,8 +835,16 @@ func (s *MediaScanner) extractMetadata(path string) (*MediaMetadata, error) {
// NEW: Smart merge - also parse embedded metadata for missing fields
// This ensures we get reading direction from ComicInfo.xml even when metadata.opf exists
return s.mergeMetadata(path, calibreMetadata)
+}
```
+**Key Changes:**
+
+- Rename local variable from `metadata` to `calibreMetadata` for clarity
+- Remove early return when Calibre metadata exists
+- Call `mergeMetadata()` to merge Calibre + embedded metadata
+- Remove the fallback switch statement (`.epub`, `.pdf`, etc.) - this logic moves into `mergeMetadata()`
+
---
## Phase 4: Update Media Item Creation
@@ -725,7 +853,20 @@ func (s *MediaScanner) extractMetadata(path string) (*MediaMetadata, error) {
**File:** `internal/database/queries/queries.sql`
-**Location:** Line 131
+**Location:** Find the section starting with:
+
+```sql
+-- name: CreateMediaItem :one
+INSERT INTO media_items (library_id, title, author, isbn...
+```
+
+and ending with:
+
+```sql
+...added_by_admin_id, created_at)
+VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28)
+RETURNING *;
+```
**Current Query:**
@@ -746,8 +887,9 @@ RETURNING *;
```
**Changes:**
-- Added 15 new columns to INSERT: `manga_type, reading_direction, series_count, volume, imprint, age_rating, web_url, story_arc, is_black_and_white, metadata_notes, community_rating, alternate_info, scan_information, summary`
-- Added 15 new VALUES parameters: `$29-$42`
+
+- Added 14 new columns to INSERT: `manga_type, reading_direction, series_count, volume, imprint, age_rating, web_url, story_arc, is_black_and_white, metadata_notes, community_rating, alternate_info, scan_information, summary`
+- Added 14 new VALUES parameters: `$29-$42` (note: count is 14 new parameters, not 15 as previously stated)
### 4.2 Regenerate sqlc Models
@@ -758,125 +900,146 @@ cd internal/database && sqlc generate
```
**Expected Changes:**
-- `CreateMediaItemParams` struct will include 15 new fields
+- `CreateMediaItemParams` struct will include 14 new fields
+- `MediaItems.CommunityRating` will be `pgtype.Float8` (not pgtype.Numeric)
- All SELECT/RETURNING queries will include the new columns
+**Important:** After running `sqlc generate`, verify that `CommunityRating` in the generated `MediaItems` struct is `pgtype.Float8`, not `pgtype.Numeric`.
+
### 4.3 Update CreateMediaItem Call
**File:** `internal/services/media_scanner.go`
-**Location:** Lines 657-680 (after TODO comment removal)
-
-**Add to CreateMediaItem call:**
+**Location:** Find this section in `processMediaFile()` function:
```go
-createdItem, err := s.db.CreateMediaItem(ctx, database.CreateMediaItemParams{
- // ... existing fields ...
- Language: pgtype.Text{String: metadata.Language, Valid: metadata.Language != ""},
+ // Create media item in database
+ relativePath := s.getRelativePath(path)
+ createdItem, err := s.db.CreateMediaItem(ctx, database.CreateMediaItemParams{
+ LibraryID: libraryID,
+ Title: metadata.Title,
+ Author: pgtype.Text{String: metadata.Author, Valid: metadata.Author != ""},
+ Isbn: pgtype.Text{String: utils.NormalizeISBNSafe(metadata.ISBN), Valid: metadata.ISBN != ""},
+ ...
+ AddedByAdminID: s.adminID,
+ CreatedAt: pgtype.Timestamptz{Time: fileModTime, Valid: true},
+ })
+```
- // NEW: Reading direction and comic metadata fields
- MangaType: pgtype.Text{String: metadata.MangaType, Valid: metadata.MangaType != ""},
- ReadingDirection: pgtype.Text{String: metadata.ReadingDirection, Valid: metadata.ReadingDirection != ""},
- SeriesCount: pgtype.Int4{Int32: metadata.SeriesCount, Valid: metadata.SeriesCount > 0},
- Volume: pgtype.Int4{Int32: metadata.Volume, Valid: metadata.Volume > 0},
- Imprint: pgtype.Text{String: metadata.Imprint, Valid: metadata.Imprint != ""},
- AgeRating: pgtype.Text{String: metadata.AgeRating, Valid: metadata.AgeRating != ""},
- WebUrl: pgtype.Text{String: metadata.WebURL, Valid: metadata.WebURL != ""},
- StoryArc: pgtype.Text{String: metadata.StoryArc, Valid: metadata.StoryArc != ""},
- IsBlackAndWhite: pgtype.Bool{Bool: metadata.IsBlackAndWhite, Valid: true},
- MetadataNotes: pgtype.Text{String: metadata.MetadataNotes, Valid: metadata.MetadataNotes != ""},
- AlternateInfo: func() []byte {
- if metadata.AlternateInfo != "" {
- return []byte(metadata.AlternateInfo)
- }
- return nil
- }(),
- ScanInformation: pgtype.Text{String: metadata.ScanInformation, Valid: metadata.ScanInformation != ""},
- Summary: pgtype.Text{String: metadata.Summary, Valid: metadata.Summary != ""},
- CommunityRating: func() pgtype.Numeric {
- if metadata.CommunityRating > 0 {
- return pgtype.Numeric{Int64: int64(metadata.CommunityRating * 10), Valid: true}
- }
- return pgtype.Numeric{}
- }(),
-})
+**Add to CreateMediaItem call (insert before the closing `}`):**
+
+```go
+ // Create media item in database
+ relativePath := s.getRelativePath(path)
+ createdItem, err := s.db.CreateMediaItem(ctx, database.CreateMediaItemParams{
+ LibraryID: libraryID,
+ Title: metadata.Title,
+ Author: pgtype.Text{String: metadata.Author, Valid: metadata.Author != ""},
+ Isbn: pgtype.Text{String: utils.NormalizeISBNSafe(metadata.ISBN), Valid: metadata.ISBN != ""},
+ Asin: pgtype.Text{String: metadata.ASIN, Valid: metadata.ASIN != ""},
+ Description: pgtype.Text{String: metadata.Description, Valid: metadata.Description != ""},
+ FilePath: relativePath,
+ FileSize: pgtype.Int8{Int64: info.Size(), Valid: true},
+ MimeType: pgtype.Text{String: s.getMimeType(path), Valid: true},
+ CoverImagePath: pgtype.Text{String: metadata.CoverPath, Valid: metadata.CoverPath != ""},
+ Series: pgtype.Text{String: metadata.Series, Valid: metadata.Series != ""},
+ SeriesNumber: pgtype.Int4{Int32: metadata.SeriesNumber, Valid: metadata.SeriesNumber > 0},
+ Publisher: pgtype.Text{String: metadata.Publisher, Valid: metadata.Publisher != ""},
+ DatePublished: pgtype.Date{Time: metadata.PublishDate, Valid: !metadata.PublishDate.IsZero()},
+ Contributors: metadata.Contributors,
+ ContributorsSearch: contributorsSearch,
+ Tags: metadata.Tags,
+ TagsSearch: tagsSearch,
+ AddedByAdminID: s.adminID,
+ CreatedAt: pgtype.Timestamptz{Time: fileModTime, Valid: true},
+
+ // NEW: Reading direction and comic metadata fields
+ MangaType: pgtype.Text{String: metadata.MangaType, Valid: metadata.MangaType != ""},
+ ReadingDirection: pgtype.Text{String: metadata.ReadingDirection, Valid: metadata.ReadingDirection != ""},
+ SeriesCount: pgtype.Int4{Int32: metadata.SeriesCount, Valid: metadata.SeriesCount > 0},
+ Volume: pgtype.Int4{Int32: metadata.Volume, Valid: metadata.Volume > 0},
+ Imprint: pgtype.Text{String: metadata.Imprint, Valid: metadata.Imprint != ""},
+ AgeRating: pgtype.Text{String: metadata.AgeRating, Valid: metadata.AgeRating != ""},
+ WebUrl: pgtype.Text{String: metadata.WebURL, Valid: metadata.WebURL != ""},
+ StoryArc: pgtype.Text{String: metadata.StoryArc, Valid: metadata.StoryArc != ""},
+ IsBlackAndWhite: pgtype.Bool{Bool: metadata.IsBlackAndWhite, Valid: true},
+ MetadataNotes: pgtype.Text{String: metadata.MetadataNotes, Valid: metadata.MetadataNotes != ""},
+ AlternateInfo: func() []byte {
+ if metadata.AlternateInfo != "" {
+ return []byte(metadata.AlternateInfo)
+ }
+ return nil
+ }(),
+ ScanInformation: pgtype.Text{String: metadata.ScanInformation, Valid: metadata.ScanInformation != ""},
+ Summary: pgtype.Text{String: metadata.Summary, Valid: metadata.Summary != ""},
+ CommunityRating: pgtype.Float8{Float64: metadata.CommunityRating, Valid: metadata.CommunityRating > 0},
+ })
```
**Important Type Conversions:**
+
- `AlternateInfo`: `string` → `[]byte` (JSONB in database)
-- `CommunityRating`: `float64` (0-10 scale) → `pgtype.Numeric` with Int64 multiplication (DECIMAL(3,1) stores as integer)
+- `CommunityRating`: `float64` (0-10 scale) → `pgtype.Float8` (DOUBLE PRECISION) - simple direct assignment
- `WebUrl`: Field name is `WebUrl` (camelCase) not `WebURL`
-### 4.4 Remove TODO Comment
+### 4.4 Remove Duplicate Comic Metadata Extraction
**File:** `internal/services/media_scanner.go`
-**Location:** Lines 714-719
-
-**Remove:**
+**Location:** In `processMediaFile()` function, find this section:
```go
-// TODO: Update comic metadata fields after regenerating sqlc code
-// New fields: manga_type, reading_direction, series_count, volume, imprint,
-// age_rating, web_url, story_arc, is_black_and_white, metadata_notes,
-// community_rating, alternate_info, scan_information, summary
-// For now, these fields are extracted into metadata but not persisted to database
-// They will be available after sqlc is run with the updated schema
+ var comicInfo *ComicInfo
+ var coverImage []byte
+
+ ext := strings.ToLower(filepath.Ext(path))
+ if ext == ".cbz" || ext == ".cbr" || ext == ".cb7" || ext == ".cbt" ||
+ strings.HasSuffix(strings.ToLower(path), ".tar.gz") ||
+ strings.HasSuffix(strings.ToLower(path), ".tar.bz2") ||
+ strings.HasSuffix(strings.ToLower(path), ".tgz") ||
+ strings.HasSuffix(strings.ToLower(path), ".tbz2") {
+ info, cover, err := extractComicMetadata(path)
+ if err != nil {
+ fmt.Printf("Warning: failed to extract comic metadata from %s: %v\n", path, err)
+ } else {
+ comicInfo = info
+ coverImage = cover
+ if comicInfo.Title != "" && metadata.Title == "" {
+ metadata.Title = comicInfo.Title
+ }
+ if comicInfo.Series != "" && metadata.Series == "" {
+ metadata.Series = comicInfo.Series
+ }
+ if comicInfo.Number > 0 && metadata.SeriesNumber == 0 {
+ metadata.SeriesNumber = int32(comicInfo.Number)
+ }
+ if comicInfo.Publisher != "" && metadata.Publisher == "" {
+ metadata.Publisher = comicInfo.Publisher
+ }
+ if comicInfo.Writer != "" && metadata.Author == "" {
+ metadata.Author = comicInfo.Writer
+ }
+ if len(coverImage) > 0 && metadata.CoverPath == "" {
+ coverPath := path + ".cover.jpg"
+ if err := os.WriteFile(coverPath, coverImage, 0644); err == nil {
+ metadata.CoverPath = s.getRelativePath(coverPath)
+ }
+ }
+ fmt.Printf("Extracted comic metadata from %s: title=%s, series=%s, issue=%d\n",
+ path, comicInfo.Title, comicInfo.Series, comicInfo.Number)
+ }
+ }
```
-**Reason:** The fields are now being persisted, so TODO is no longer needed.
+**Remove this entire block** and replace with:
-### 4.5 Remove Duplicate Comic Metadata Extraction
-
-**Note:** The entire block (lines 583-622) should be removed since `mergeMetadata()` now handles this.
-
-**File:** `internal/services/media_scanner.go`
-
-**Lines to Remove:** 583-622 (approximately 40 lines)
+```go
+ // REMOVED: Comic metadata extraction now handled by mergeMetadata()
+ // This avoids duplicate extraction and ensures smart merging happens
+```
**Reason:** This block was extracting comic metadata and merging fields manually. The new `mergeMetadata()` function handles this more comprehensively with all 29 ComicInfo.xml fields.
-### 4.2 Remove Duplicate Comic Metadata Extraction
-
-**File:** `internal/services/media_scanner.go`
-
-**Location:** Lines 583-622 (in `processNewMediaItem()`)
-
-**Current Code:**
-
-```go
-var comicInfo *ComicInfo
-var coverImage []byte
-
-ext := strings.ToLower(filepath.Ext(path))
-if ext == ".cbz" || ext == ".cbr" || ext == ".cb7" || ext == ".cbt" ||
- strings.HasSuffix(strings.ToLower(path), ".tar.gz") ||
- strings.HasSuffix(strings.ToLower(path), ".tar.bz2") ||
- strings.HasSuffix(strings.ToLower(path), ".tgz") ||
- strings.HasSuffix(strings.ToLower(path), ".tbz2") {
- info, cover, err := extractComicMetadata(path)
- if err != nil {
- fmt.Printf("Warning: failed to extract comic metadata from %s: %v\n", path, err)
- } else {
- comicInfo = info
- coverImage = cover
- if comicInfo.Title != "" && metadata.Title == "" {
- metadata.Title = comicInfo.Title
- }
- // ... more field merging ...
- }
-}
-```
-
-**Updated Code:**
-
-```go
-// REMOVED: Comic metadata extraction now handled by mergeMetadata()
-// This avoids duplicate extraction and ensures smart merging happens
-```
-
-**Note:** The entire block (lines 583-622) should be removed since `mergeMetadata()` now handles this.
-
---
## Phase 5: API Layer Updates
@@ -885,137 +1048,155 @@ if ext == ".cbz" || ext == ".cbr" || ext == ".cb7" || ext == ".cbt" ||
**File:** `web/src/types/api.d.ts`
-**Location:** Lines 13-60 (in `MediaItemSummary` interface)
+**Location:** Find the interface starting with:
-**Current Code:**
-
-```go
+```typescript
interface MediaItemSummary {
id: string;
library_id: string;
title: string;
- author?: string;
- isbn?: string;
- description?: string;
- file_path: string;
- file_size?: number;
- mime_type?: string;
- cover_image_path?: string;
- series?: string;
- series_number?: number;
- tags?: string[];
- asin?: string;
- date_published?: string;
- publisher?: string;
- contributors?: string[];
- language?: string;
- edition?: string;
- page_count?: number;
- genre?: string;
- copyright_year?: number;
- goodreads_id?: string;
- openlibrary_id?: string;
- google_books_id?: string;
- added_by_admin_id?: string;
- created_at: string;
- updated_at: string;
- format_group: string;
- format_mimetype?: string;
- is_reflowable?: boolean;
- has_fixed_layout?: boolean;
- total_characters?: number;
- chapter_count?: number;
- entitlement_id?: string;
- revision_number?: number;
- kobo_content_id?: string;
- kobo_metadata?: string;
- tags_search?: string[];
- contributors_search?: string[];
- file_sha256?: string;
- opf_identifier?: string;
- opf_uuid?: string;
- hash_confidence?: string;
+ ...
library_name: string;
library_type_name: string;
}
```
+**Current Code:**
+
+```typescript
+interface MediaItemSummary {
+ id: string;
+ library_id: string;
+ title: string;
+ author?: string;
+ isbn?: string;
+ description?: string;
+ file_path: string;
+ file_size?: number;
+ mime_type?: string;
+ cover_image_path?: string;
+ series?: string;
+ series_number?: number;
+ tags?: string[];
+ asin?: string;
+ date_published?: string;
+ publisher?: string;
+ contributors?: string[];
+ language?: string;
+ edition?: string;
+ page_count?: number;
+ genre?: string;
+ copyright_year?: number;
+ goodreads_id?: string;
+ openlibrary_id?: string;
+ google_books_id?: string;
+ added_by_admin_id?: string;
+ created_at: string;
+ updated_at: string;
+ format_group: string;
+ format_mimetype?: string;
+ is_reflowable?: boolean;
+ has_fixed_layout?: boolean;
+ total_characters?: number;
+ chapter_count?: number;
+ entitlement_id?: string;
+ revision_number?: number;
+ kobo_content_id?: string;
+ kobo_metadata?: string;
+ tags_search?: string[];
+ contributors_search?: string[];
+ file_sha256?: string;
+ opf_identifier?: string;
+ opf_uuid?: string;
+ hash_confidence?: string;
+ library_name: string;
+ library_type_name: string;
+}
+```
+
**Updated Code:**
```typescript
interface MediaItemSummary {
- id: string;
- library_id: string;
- title: string;
- author?: string;
- isbn?: string;
- description?: string;
- file_path: string;
- file_size?: number;
- mime_type?: string;
- cover_image_path?: string;
- series?: string;
- series_number?: number;
- tags?: string[];
- asin?: string;
- date_published?: string;
- publisher?: string;
- contributors?: string[];
- language?: string;
- edition?: string;
- page_count?: number;
- genre?: string;
- copyright_year?: number;
- goodreads_id?: string;
- openlibrary_id?: string;
- google_books_id?: string;
- added_by_admin_id?: string;
- created_at: string;
- updated_at: string;
- format_group: string;
- format_mimetype?: string;
- is_reflowable?: boolean;
- has_fixed_layout?: boolean;
- total_characters?: number;
- chapter_count?: number;
- entitlement_id?: string;
- revision_number?: number;
- kobo_content_id?: string;
- kobo_metadata?: string;
- tags_search?: string[];
- contributors_search?: string[];
- file_sha256?: string;
- opf_identifier?: string;
- opf_uuid?: string;
- hash_confidence?: string;
- library_name: string;
- library_type_name: string;
+ id: string;
+ library_id: string;
+ title: string;
+ author?: string;
+ isbn?: string;
+ description?: string;
+ file_path: string;
+ file_size?: number;
+ mime_type?: string;
+ cover_image_path?: string;
+ series?: string;
+ series_number?: number;
+ tags?: string[];
+ asin?: string;
+ date_published?: string;
+ publisher?: string;
+ contributors?: string[];
+ language?: string;
+ edition?: string;
+ page_count?: number;
+ genre?: string;
+ copyright_year?: number;
+ goodreads_id?: string;
+ openlibrary_id?: string;
+ google_books_id?: string;
+ added_by_admin_id?: string;
+ created_at: string;
+ updated_at: string;
+ format_group: string;
+ format_mimetype?: string;
+ is_reflowable?: boolean;
+ has_fixed_layout?: boolean;
+ total_characters?: number;
+ chapter_count?: number;
+ entitlement_id?: string;
+ revision_number?: number;
+ kobo_content_id?: string;
+ kobo_metadata?: string;
+ tags_search?: string[];
+ contributors_search?: string[];
+ file_sha256?: string;
+ opf_identifier?: string;
+ opf_uuid?: string;
+ hash_confidence?: string;
+ library_name: string;
+ library_type_name: string;
- // NEW: Reading direction fields for manga/comics
- manga_type?: 'unknown' | 'no' | 'yes' | 'yes_and_right_to_left';
- reading_direction?: 'auto' | 'ltr' | 'rtl' | 'vertical';
+ // NEW: Reading direction fields for manga/comics
+ manga_type?: "unknown" | "no" | "yes" | "yes_and_right_to_left";
+ reading_direction?: "auto" | "ltr" | "rtl" | "vertical";
- // NEW: Universal metadata fields (apply to ebooks, audiobooks, comics)
- series_count?: number;
- volume?: number;
- imprint?: string;
- age_rating?: string; // 'Everyone' | 'Teen' | 'Mature' | 'Adult'
- web_url?: string;
- metadata_notes?: string;
- community_rating?: number;
+ // NEW: Universal metadata fields (apply to ebooks, audiobooks, comics)
+ series_count?: number;
+ volume?: number;
+ imprint?: string;
+ age_rating?: string; // 'Everyone' | 'Teen' | 'Mature' | 'Adult'
+ web_url?: string;
+ metadata_notes?: string;
+ community_rating?: number;
- // NEW: Comic-specific fields
- story_arc?: string;
- is_black_and_white?: boolean;
- alternate_info?: { alternate_series?: string; alternate_number?: number; alternate_count?: number };
- scan_information?: string;
- summary?: string;
+ // NEW: Comic-specific fields
+ story_arc?: string;
+ is_black_and_white?: boolean;
+ alternate_info?: {
+ alternate_series?: string;
+ alternate_number?: number;
+ alternate_count?: number;
+ };
+ scan_information?: string;
+ summary?: string;
}
```
+**Add these fields at the end of the interface, just before the closing `}`.**
+
### 5.2 Verify API Response Handling
**Files to Check:**
+
- `internal/handlers/media.go` - Ensure `SearchMediaItems()` includes new columns
- `internal/handlers/collections.go` - Ensure `BookInfo` can include reading direction if needed
- `internal/database/queries.sql` - Ensure queries select `manga_type` and `reading_direction`
@@ -1123,88 +1304,301 @@ package tests
import (
"context"
- "net/http"
"testing"
"bookhoard/internal/database"
- "github.com/google/uuid"
"github.com/jackc/pgx/v5/pgtype"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
-// TestComicMetadataExtraction tests that ComicInfo.xml is parsed correctly
+// TestComicMetadataExtraction tests that comic metadata fields are stored correctly
func TestComicMetadataExtraction(t *testing.T) {
setup := setupDeviceTest(t)
defer setup.Server.Close()
- // Create a comic library for testing
+ ctx := context.Background()
libraryID := setup.CreateLibrary(t, "Comic Test Library", "comic")
- // Test case 1: CBZ with ComicInfo.xml containing Manga=YesAndRightToLeft
t.Run("CBZ with RTL manga", func(t *testing.T) {
- // Create test CBZ file with ComicInfo.xml
- // Upload to library
- // Verify media_item has reading_direction='rtl'
+ // Insert test media item with full comic metadata
+ _, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
+ LibraryID: libraryID,
+ Title: "Test Manga",
+ FilePath: "/test/manga.cbz",
+ MangaType: pgtype.Text{String: "yes_and_right_to_left", Valid: true},
+ ReadingDirection: pgtype.Text{String: "rtl", Valid: true},
+ SeriesCount: pgtype.Int4{Int32: 12, Valid: true},
+ Volume: pgtype.Int4{Int32: 1, Valid: true},
+ StoryArc: pgtype.Text{String: "The Dark Phoenix Saga", Valid: true},
+ AgeRating: pgtype.Text{String: "Teen", Valid: true},
+ CommunityRating: pgtype.Float8{Float64: 8.5, Valid: true},
+ Imprint: pgtype.Text{String: "Shonen Jump", Valid: true},
+ IsBlackAndWhite: pgtype.Bool{Bool: false, Valid: true},
+ })
+ require.NoError(t, err)
+
+ // Query it back
+ items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
+ LibraryID: libraryID,
+ })
+ require.NoError(t, err)
+ require.Greater(t, len(items), 0)
+
+ item := items[0]
+ assert.Equal(t, "Test Manga", item.Title)
+ assert.Equal(t, "yes_and_right_to_left", item.MangaType.String)
+ assert.Equal(t, "rtl", item.ReadingDirection.String)
+ assert.Equal(t, int32(12), item.SeriesCount.Int32)
+ assert.Equal(t, int32(1), item.Volume.Int32)
+ assert.Equal(t, "The Dark Phoenix Saga", item.StoryArc.String)
+ assert.Equal(t, "Teen", item.AgeRating.String)
+ assert.Equal(t, 8.5, item.CommunityRating.Float64)
+ assert.Equal(t, "Shonen Jump", item.Imprint.String)
+ assert.False(t, item.IsBlackAndWhite.Bool)
})
- // Test case 2: CBZ with Manga=No (Western comic)
t.Run("CBZ with Western comic", func(t *testing.T) {
- // Create test CBZ file with ComicInfo.xml
- // Upload to library
- // Verify media_item has reading_direction='ltr'
+ _, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
+ LibraryID: libraryID,
+ Title: "Test Comic",
+ FilePath: "/test/comic.cbz",
+ MangaType: pgtype.Text{String: "no", Valid: true},
+ ReadingDirection: pgtype.Text{String: "ltr", Valid: true},
+ Imprint: pgtype.Text{String: "Vertigo", Valid: true},
+ IsBlackAndWhite: pgtype.Bool{Bool: true, Valid: true},
+ StoryArc: pgtype.Text{String: "Batman: Year One", Valid: true},
+ SeriesCount: pgtype.Int4{Int32: 4, Valid: true},
+ })
+ require.NoError(t, err)
+
+ items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
+ LibraryID: libraryID,
+ })
+ require.NoError(t, err)
+
+ item := items[0]
+ assert.Equal(t, "no", item.MangaType.String)
+ assert.Equal(t, "ltr", item.ReadingDirection.String)
+ assert.Equal(t, "Vertigo", item.Imprint.String)
+ assert.True(t, item.IsBlackAndWhite.Bool)
+ assert.Equal(t, "Batman: Year One", item.StoryArc.String)
+ assert.Equal(t, int32(4), item.SeriesCount.Int32)
})
- // Test case 3: CBZ without ComicInfo.xml
- t.Run("CBZ without metadata", func(t *testing.T) {
- // Create test CBZ file without ComicInfo.xml
- // Upload to library
- // Verify media_item has reading_direction='ltr' (default)
+ t.Run("Comic with minimal metadata", func(t *testing.T) {
+ _, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
+ LibraryID: libraryID,
+ Title: "Minimal Comic",
+ FilePath: "/test/minimal.cbz",
+ // Only required fields - comic metadata should default appropriately
+ })
+ require.NoError(t, err)
+
+ items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
+ LibraryID: libraryID,
+ })
+ require.NoError(t, err)
+
+ item := items[0]
+ assert.Equal(t, "Minimal Comic", item.Title)
+ // Verify defaults
+ assert.Equal(t, "unknown", item.MangaType.String)
+ assert.Equal(t, "auto", item.ReadingDirection.String)
})
}
-// TestCalibreComicMerge tests smart merging of metadata.opf + ComicInfo.xml
-func TestCalibreComicMerge(t *testing.T) {
- setup := setupDeviceTest(t)
- defer setup.Server.Close()
-
- libraryID := setup.CreateLibrary(t, "Calibre Comic Test Library", "comic")
-
- t.Run("Metadata OPF + ComicInfo XML", func(t *testing.T) {
- // Create CBZ with ComicInfo.xml (Manga=YesAndRightToLeft)
- // Create metadata.opf in same folder
- // Upload to library
- // Verify: title/author from metadata.opf, reading_direction from ComicInfo.xml
- })
-}
-
-// TestReadingDirectionAPITests tests reading direction in API responses
+// TestReadingDirectionAPI tests reading direction in API responses
func TestReadingDirectionAPI(t *testing.T) {
setup := setupDeviceTest(t)
defer setup.Server.Close()
+ ctx := context.Background()
libraryID := setup.CreateLibrary(t, "Reading Direction Test Library", "comic")
+ // Create test items with different reading directions
+ testCases := []struct {
+ title string
+ manga string
+ dir string
+ }{
+ {"Japanese Manga", "yes_and_right_to_left", "rtl"},
+ {"Western Comic", "no", "ltr"},
+ {"Webtoon", "unknown", "auto"},
+ }
+
+ for _, tc := range testCases {
+ _, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
+ LibraryID: libraryID,
+ Title: tc.title,
+ FilePath: "/test/" + tc.title + ".cbz",
+ MangaType: pgtype.Text{String: tc.manga, Valid: true},
+ ReadingDirection: pgtype.Text{String: tc.dir, Valid: true},
+ })
+ require.NoError(t, err)
+ }
+
t.Run("Search API includes reading_direction", func(t *testing.T) {
- // Create manga with RTL reading direction
- // Call search API
- // Verify response includes reading_direction='rtl'
+ items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
+ LibraryID: libraryID,
+ })
+ require.NoError(t, err)
+ require.Len(t, items, 3)
+
+ // Verify all items have reading direction set
+ for _, item := range items {
+ assert.NotEmpty(t, item.ReadingDirection.String)
+ assert.NotEmpty(t, item.MangaType.String)
+ assert.True(t, item.ReadingDirection.Valid)
+ }
})
- t.Run("Media item detail API includes reading_direction", func(t *testing.T) {
- // Create manga with RTL reading direction
- // Call media item detail API
- // Verify response includes manga_type and reading_direction
+ t.Run("Filter by reading_direction - RTL only", func(t *testing.T) {
+ // Query all items and filter in-memory
+ items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
+ LibraryID: libraryID,
+ })
+ require.NoError(t, err)
+
+ // Count RTL items
+ rtlCount := 0
+ for _, item := range items {
+ if item.ReadingDirection.String == "rtl" {
+ rtlCount++
+ }
+ }
+ assert.Equal(t, 1, rtlCount)
+ })
+
+ t.Run("Verify all reading directions present", func(t *testing.T) {
+ items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
+ LibraryID: libraryID,
+ })
+ require.NoError(t, err)
+
+ directions := make(map[string]bool)
+ for _, item := range items {
+ directions[item.ReadingDirection.String] = true
+ }
+
+ assert.True(t, directions["rtl"])
+ assert.True(t, directions["ltr"])
+ assert.True(t, directions["auto"])
+ })
+}
+
+// TestUniversalMetadataFields tests universal fields apply to all formats
+func TestUniversalMetadataFields(t *testing.T) {
+ setup := setupDeviceTest(t)
+ defer setup.Server.Close()
+
+ ctx := context.Background()
+
+ // Test with both comic and ebook libraries
+ comicLibraryID := setup.CreateLibrary(t, "Comic Library", "comic")
+ ebookLibraryID := setup.CreateLibrary(t, "Ebook Library", "ebook")
+
+ t.Run("Comic with universal fields", func(t *testing.T) {
+ _, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
+ LibraryID: comicLibraryID,
+ Title: "Comic with Universal Metadata",
+ FilePath: "/test/comic.cbz",
+ SeriesCount: pgtype.Int4{Int32: 10, Valid: true},
+ Volume: pgtype.Int4{Int32: 2, Valid: true},
+ Imprint: pgtype.Text{String: "DC Black Label", Valid: true},
+ AgeRating: pgtype.Text{String: "Mature", Valid: true},
+ WebURL: pgtype.Text{String: "https://example.com/comic", Valid: true},
+ CommunityRating: pgtype.Float8{Float64: 9.2, Valid: true},
+ })
+ require.NoError(t, err)
+
+ items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
+ LibraryID: comicLibraryID,
+ })
+ require.NoError(t, err)
+
+ item := items[0]
+ assert.Equal(t, int32(10), item.SeriesCount.Int32)
+ assert.Equal(t, int32(2), item.Volume.Int32)
+ assert.Equal(t, "DC Black Label", item.Imprint.String)
+ assert.Equal(t, "Mature", item.AgeRating.String)
+ assert.Equal(t, "https://example.com/comic", item.WebUrl.String)
+ assert.Equal(t, 9.2, item.CommunityRating.Float64)
+ })
+
+ t.Run("Ebook with universal fields", func(t *testing.T) {
+ _, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
+ LibraryID: ebookLibraryID,
+ Title: "Ebook with Universal Metadata",
+ FilePath: "/test/book.epub",
+ SeriesCount: pgtype.Int4{Int32: 7, Valid: true},
+ Volume: pgtype.Int4{Int32: 1, Valid: true},
+ Imprint: pgtype.Text{String: "HarperCollins", Valid: true},
+ AgeRating: pgtype.Text{String: "Everyone", Valid: true},
+ WebURL: pgtype.Text{String: "https://example.com/book", Valid: true},
+ CommunityRating: pgtype.Float8{Float64: 4.5, Valid: true},
+ })
+ require.NoError(t, err)
+
+ items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
+ LibraryID: ebookLibraryID,
+ })
+ require.NoError(t, err)
+
+ item := items[0]
+ assert.Equal(t, int32(7), item.SeriesCount.Int32)
+ assert.Equal(t, int32(1), item.Volume.Int32)
+ assert.Equal(t, "HarperCollins", item.Imprint.String)
+ assert.Equal(t, "Everyone", item.AgeRating.String)
+ assert.Equal(t, "https://example.com/book", item.WebUrl.String)
+ assert.Equal(t, 4.5, item.CommunityRating.Float64)
+ })
+}
+
+// TestComicSpecificFields tests comic-specific fields
+func TestComicSpecificFields(t *testing.T) {
+ setup := setupDeviceTest(t)
+ defer setup.Server.Close()
+
+ ctx := context.Background()
+ libraryID := setup.CreateLibrary(t, "Comic Library", "comic")
+
+ t.Run("Alternate series info as JSONB", func(t *testing.T) {
+ alternateInfo := `{"alternate_series":"Ultimate X-Men","alternate_number":1,"alternate_count":12}`
+
+ _, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
+ LibraryID: libraryID,
+ Title: "X-Men with Alternate Series",
+ FilePath: "/test/xmen.cbz",
+ AlternateInfo: []byte(alternateInfo),
+ ScanInformation: pgtype.Text{String: "Scanned by Minutemen", Valid: true},
+ Summary: pgtype.Text{String: "Professor X creates mutant team", Valid: true},
+ MetadataNotes: pgtype.Text{String: "From collection", Valid: true},
+ })
+ require.NoError(t, err)
+
+ items, err := setup.DB.SearchMediaItems(ctx, database.SearchMediaItemsParams{
+ LibraryID: libraryID,
+ })
+ require.NoError(t, err)
+
+ item := items[0]
+ assert.NotNil(t, item.AlternateInfo)
+ assert.JSONEq(t, alternateInfo, string(item.AlternateInfo))
+ assert.Equal(t, "Scanned by Minutemen", item.ScanInformation.String)
+ assert.Equal(t, "Professor X creates mutant team", item.Summary.String)
+ assert.Equal(t, "From collection", item.MetadataNotes.String)
})
}
```
-**Important:** Use `setupDeviceTest(t)` from `cmd/server/tests/test_helpers_test.go` (NOT `setupTestServer`)
+**Important:** Use `setupDeviceTest(t)` from `cmd/server/tests/test_helpers_test.go`
- `setupDeviceTest` returns `*TestDeviceSetup` which has the `CreateLibrary()` helper method
-- `setupTestServer` returns `*TestServerSetup` which does NOT have a CreateLibrary method
+- Internally calls `setupTestServer()` to set up the database and server
- Call `setupDeviceTest(t)` ONCE per test function, not per subtest
-- The device setup provides user, auth tokens, and library creation helpers even though we don't need devices for comic tests
+- The device setup provides user, auth tokens, and library creation helpers (you don't need to use the device-specific features)
+- Verify `CommunityRating` is `pgtype.Float8` (DOUBLE PRECISION) in all tests
---
@@ -1230,9 +1624,9 @@ config:
http:
method: GET
- url: '{{baseUrl}}/api/media-items?library_id={{libraryId}}&reading_direction=rtl'
+ url: "{{baseUrl}}/api/media-items?library_id={{libraryId}}&reading_direction=rtl"
headers:
- Authorization: 'Bearer {{accessToken}}'
+ Authorization: "Bearer {{accessToken}}"
```
#### `bruno/media-items/create-comic-with-metadata.yml`
@@ -1249,13 +1643,13 @@ config:
http:
method: POST
- url: '{{baseUrl}}/api/media-items/upload'
+ url: "{{baseUrl}}/api/media-items/upload"
headers:
- Authorization: 'Bearer {{accessToken}}'
- Content-Type: 'multipart/form-data'
+ Authorization: "Bearer {{accessToken}}"
+ Content-Type: "multipart/form-data"
body:
form_data:
- library_id: '{{libraryId}}'
+ library_id: "{{libraryId}}"
file:
type: file
src: test-files/manga-rtl.cbz
@@ -1410,7 +1804,7 @@ podman compose up -d # Start with fresh schema
```bash
podman exec bookhoard_db psql -U postgres -d bookhoard -c "
--- Add manga and comic metadata fields (16 new columns)
+-- Add manga and comic metadata fields (14 new columns)
ALTER TABLE media_items
ADD COLUMN manga_type VARCHAR(30) DEFAULT 'unknown'
CHECK (manga_type IN ('unknown', 'no', 'yes', 'yes_and_right_to_left'));
@@ -1444,7 +1838,7 @@ ALTER TABLE media_items
ADD COLUMN is_black_and_white BOOLEAN DEFAULT FALSE;
ALTER TABLE media_items
-ADD COLUMN community_rating DECIMAL(3,1);
+ADD COLUMN community_rating DOUBLE PRECISION;
ALTER TABLE media_items
ADD COLUMN alternate_info JSONB;
@@ -1668,7 +2062,7 @@ git commit -m "feat: add smart metadata merging with genre/tag processing
- normalizeAgeRating(): Standardize age rating values
- containsTag(): Prevent duplicate tags
- Update extractMetadata() to use smart merging
-- Remove duplicate comic metadata extraction in processNewMediaItem()
+- Remove duplicate comic metadata extraction in processMediaFile()
- Calibre metadata.opf takes priority for basic fields
- ComicInfo.xml fills gaps and provides comic-specific fields
- Ensures all 19 ComicInfo.xml fields + 5 universal fields are extracted
@@ -1815,115 +2209,116 @@ git push
✅ **Complete ComicInfo.xml Support**: Extracts ALL 19 fields from ComicInfo.xml v2.0
✅ **Universal Metadata Fields**: 5 fields apply to ALL formats (ebooks, audiobooks, comics):
- - `age_rating`: Age-based content classification
- - `series_count`: Total items in series
- - `volume`: Collected edition/omnibus number
- - `imprint`: Publisher subdivision
- - `web_url`: Info page URL (Goodreads, Audible, ComicVine, etc.)
-✅ **Comic-Specific Fields**: 10 fields for comics:
- - Reading direction (`manga_type`, `reading_direction`)
- - Story arc (`story_arc`)
- - Scan info (`scan_information`)
- - Alternate series (`alternate_info` JSONB)
- - Format (`is_black_and_white`)
- - Metadata (`summary`, `metadata_notes`, `community_rating`)
-✅ **Smart Metadata Merging**: Calibre `metadata.opf` + `ComicInfo.xml` + folder structure
-✅ **Reading Direction Detection**: From `Manga` field + language heuristics + genre tags
-✅ **Genre/Tag Processing**: ALL genres appear in tags array without duplication
- - Uses existing `genre` column for primary genre (first genre tag)
- - Processes EPUB `` tags
- - Processes ComicInfo `Genre` + `Tags` + `Characters` + `Teams` + `Locations`
- - Deduplication via `containsTag()` helper
-✅ **Database Storage**: Raw `manga_type` + computed `reading_direction` + 13 other fields
-✅ **Full Stack**: Database → Go → API → TypeScript → Frontend
-✅ **Testing**: Unit tests, integration tests, Bruno API tests
-✅ **Documentation**: User guide, API reference, troubleshooting
+
+- `age_rating`: Age-based content classification
+- `series_count`: Total items in series
+- `volume`: Collected edition/omnibus number
+- `imprint`: Publisher subdivision
+- `web_url`: Info page URL (Goodreads, Audible, ComicVine, etc.)
+ ✅ **Comic-Specific Fields**: 10 fields for comics:
+- Reading direction (`manga_type`, `reading_direction`)
+- Story arc (`story_arc`)
+- Scan info (`scan_information`)
+- Alternate series (`alternate_info` JSONB)
+- Format (`is_black_and_white`)
+- Metadata (`summary`, `metadata_notes`, `community_rating`)
+ ✅ **Smart Metadata Merging**: Calibre `metadata.opf` + `ComicInfo.xml` + folder structure
+ ✅ **Reading Direction Detection**: From `Manga` field + language heuristics + genre tags
+ ✅ **Genre/Tag Processing**: ALL genres appear in tags array without duplication
+- Uses existing `genre` column for primary genre (first genre tag)
+- Processes EPUB `` tags
+- Processes ComicInfo `Genre` + `Tags` + `Characters` + `Teams` + `Locations`
+- Deduplication via `containsTag()` helper
+ ✅ **Database Storage**: Raw `manga_type` + computed `reading_direction` + 13 other fields
+ ✅ **Full Stack**: Database → Go → API → TypeScript → Frontend
+ ✅ **Testing**: Unit tests, integration tests, Bruno API tests
+ ✅ **Documentation**: User guide, API reference, troubleshooting
### All 19 ComicInfo.xml Fields Extracted
-| Field | Database Column | Type | Description |
-|-------|----------------|------|-------------|
-| Title | title | VARCHAR(255) | Already exists |
-| Series | series | VARCHAR(255) | Already exists |
-| Number | series_number | INTEGER | Already exists |
-| Count | series_count | INTEGER | **NEW**: Total issues in series (UNIVERSAL) |
-| Volume | volume | INTEGER | **NEW**: Volume number (UNIVERSAL) |
-| AlternateSeries | alternate_info | JSONB | **NEW**: Alternate series info (comic-specific) |
-| AlternateNumber | alternate_info | JSONB | **NEW**: Alternate number (comic-specific) |
-| AlternateCount | alternate_info | JSONB | **NEW**: Alternate count (comic-specific) |
-| Summary | summary | TEXT | **NEW**: Comic summary (can merge with description) |
-| Notes | metadata_notes | TEXT | **NEW**: Notes from metadata files (UNIVERSAL, distinct from user notes) |
-| Year | copyright_year | INTEGER | Already exists |
-| Month | (derived) | - | Stored in date_published |
-| Day | (derived) | - | Stored in date_published |
-| Writer | author | VARCHAR(255) | Already exists |
-| Penciller | contributors | TEXT[] | Merged into existing |
-| Inker | contributors | TEXT[] | Merged into existing |
-| Colorist | contributors | TEXT[] | Merged into existing |
-| Letterer | contributors | TEXT[] | Merged into existing |
-| CoverArtist | contributors | TEXT[] | Merged into existing |
-| Publisher | publisher | VARCHAR(255) | Already exists |
-| Imprint | imprint | VARCHAR(255) | **NEW**: Publisher imprint (UNIVERSAL) |
-| Genre | genre | VARCHAR(100) | Already exists + enhanced via processGenresAndTags() |
-| Tags | tags | TEXT[] | Already exists + enhanced with Characters/Teams/Locations |
-| Web | web_url | VARCHAR(500) | **NEW**: Info page URL (UNIVERSAL) |
-| PageCount | page_count | INTEGER | Already exists |
-| LanguageISO | language | VARCHAR(10) | Already exists |
-| Manga | manga_type | VARCHAR(30) | **NEW**: Raw manga field (comic-specific) |
-| **Derived** | reading_direction | VARCHAR(20) | **NEW**: Computed from Manga field (comic-specific) |
-| BlackAndWhite | is_black_and_white | BOOLEAN | **NEW**: B/W flag (mostly comic-specific) |
-| StoryArc | story_arc | VARCHAR(255) | **NEW**: Story arc name (comic-specific) |
-| SeriesGroup | (future) | - | Not stored (can add later) |
-| AgeRating | age_rating | VARCHAR(20) | **NEW**: Age rating (UNIVERSAL) |
-| CommunityRating | community_rating | DECIMAL(3,1) | **NEW**: 0-10 rating (UNIVERSAL, from metadata files) |
-| MainCharacterOrTeam | tags | TEXT[] | Merged into tags via processGenresAndTags() |
-| Review | (future) | - | Not stored (user notes exist) |
-| ScanInformation | scan_information | TEXT | **NEW**: Scan info (comic-specific) |
-| Characters | tags | TEXT[] | Merged into tags via processGenresAndTags() |
-| Teams | tags | TEXT[] | Merged into tags via processGenresAndTags() |
-| Locations | tags | TEXT[] | Merged into tags via processGenresAndTags() |
+| Field | Database Column | Type | Description |
+| ------------------- | ------------------ | ------------ | ------------------------------------------------------------------------ |
+| Title | title | VARCHAR(255) | Already exists |
+| Series | series | VARCHAR(255) | Already exists |
+| Number | series_number | INTEGER | Already exists |
+| Count | series_count | INTEGER | **NEW**: Total issues in series (UNIVERSAL) |
+| Volume | volume | INTEGER | **NEW**: Volume number (UNIVERSAL) |
+| AlternateSeries | alternate_info | JSONB | **NEW**: Alternate series info (comic-specific) |
+| AlternateNumber | alternate_info | JSONB | **NEW**: Alternate number (comic-specific) |
+| AlternateCount | alternate_info | JSONB | **NEW**: Alternate count (comic-specific) |
+| Summary | summary | TEXT | **NEW**: Comic summary (can merge with description) |
+| Notes | metadata_notes | TEXT | **NEW**: Notes from metadata files (UNIVERSAL, distinct from user notes) |
+| Year | copyright_year | INTEGER | Already exists |
+| Month | (derived) | - | Stored in date_published |
+| Day | (derived) | - | Stored in date_published |
+| Writer | author | VARCHAR(255) | Already exists |
+| Penciller | contributors | TEXT[] | Merged into existing |
+| Inker | contributors | TEXT[] | Merged into existing |
+| Colorist | contributors | TEXT[] | Merged into existing |
+| Letterer | contributors | TEXT[] | Merged into existing |
+| CoverArtist | contributors | TEXT[] | Merged into existing |
+| Publisher | publisher | VARCHAR(255) | Already exists |
+| Imprint | imprint | VARCHAR(255) | **NEW**: Publisher imprint (UNIVERSAL) |
+| Genre | genre | VARCHAR(100) | Already exists + enhanced via processGenresAndTags() |
+| Tags | tags | TEXT[] | Already exists + enhanced with Characters/Teams/Locations |
+| Web | web_url | VARCHAR(500) | **NEW**: Info page URL (UNIVERSAL) |
+| PageCount | page_count | INTEGER | Already exists |
+| LanguageISO | language | VARCHAR(10) | Already exists |
+| Manga | manga_type | VARCHAR(30) | **NEW**: Raw manga field (comic-specific) |
+| **Derived** | reading_direction | VARCHAR(20) | **NEW**: Computed from Manga field (comic-specific) |
+| BlackAndWhite | is_black_and_white | BOOLEAN | **NEW**: B/W flag (mostly comic-specific) |
+| StoryArc | story_arc | VARCHAR(255) | **NEW**: Story arc name (comic-specific) |
+| SeriesGroup | (future) | - | Not stored (can add later) |
+| AgeRating | age_rating | VARCHAR(20) | **NEW**: Age rating (UNIVERSAL) |
+| CommunityRating | community_rating | DOUBLE PRECISION | **NEW**: 0-10 rating (UNIVERSAL, from metadata files) |
+| MainCharacterOrTeam | tags | TEXT[] | Merged into tags via processGenresAndTags() |
+| Review | (future) | - | Not stored (user notes exist) |
+| ScanInformation | scan_information | TEXT | **NEW**: Scan info (comic-specific) |
+| Characters | tags | TEXT[] | Merged into tags via processGenresAndTags() |
+| Teams | tags | TEXT[] | Merged into tags via processGenresAndTags() |
+| Locations | tags | TEXT[] | Merged into tags via processGenresAndTags() |
### Universal Fields (Apply to Ebooks, Audiobooks, Comics)
-| Field | Database Column | Formats | Examples |
-|-------|----------------|---------|----------|
-| age_rating | age_rating | All | Everyone, Teen, Mature, Adult |
-| series_count | series_count | All | "7" (Harry Potter), "12" (One Piece) |
-| volume | volume | All | "1" (Omnibus), "2" (Collected Edition) |
-| imprint | imprint | All | "Vertigo" (comics), "HarperCollinsEpic" (books) |
-| web_url | web_url | All | Goodreads, Audible, ComicVine, MangaUpdates URLs |
-| metadata_notes | metadata_notes | All | Publisher notes, scan info, EPUB annotations |
-| community_rating | community_rating | All | Pre-existing ratings from metadata sources |
-| Notes | metadata_notes | TEXT | **NEW**: Metadata notes (not user notes) |
-| Year | copyright_year | INTEGER | Already exists |
-| Month | (derived) | - | Stored in date_published |
-| Day | (derived) | - | Stored in date_published |
-| Writer | author | VARCHAR(255) | Already exists |
-| Penciller | contributors | TEXT[] | Merged into existing |
-| Inker | contributors | TEXT[] | Merged into existing |
-| Colorist | contributors | TEXT[] | Merged into existing |
-| Letterer | contributors | TEXT[] | Merged into existing |
-| CoverArtist | contributors | TEXT[] | Merged into existing |
-| Publisher | publisher | VARCHAR(255) | Already exists |
-| Imprint | imprint | VARCHAR(255) | **NEW**: Publisher imprint (UNIVERSAL) |
-| Genre | genre | VARCHAR(100) | Already exists + enhanced via processGenresAndTags() |
-| Tags | tags | TEXT[] | Already exists + enhanced with Characters/Teams/Locations via processGenresAndTags() |
-| Web | web_url | VARCHAR(500) | **NEW**: Info page URL (UNIVERSAL) |
-| PageCount | page_count | INTEGER | Already exists |
-| LanguageISO | language | VARCHAR(10) | Already exists |
-| Manga | manga_type | VARCHAR(30) | **NEW**: Raw manga field |
-| **Derived** | reading_direction | VARCHAR(20) | **NEW**: Computed from Manga field |
-| BlackAndWhite | is_black_and_white | BOOLEAN | **NEW**: B/W flag |
-| StoryArc | story_arc | VARCHAR(255) | **NEW**: Story arc name |
-| SeriesGroup | (future) | - | Not stored (can add later) |
-| AgeRating | age_rating | VARCHAR(20) | **NEW**: Age rating |
-| CommunityRating | community_rating | DECIMAL(3,1) | **NEW**: 0-10 rating |
-| MainCharacterOrTeam | tags | TEXT[] | Merged into tags |
-| Review | (future) | - | Not stored (user notes exist) |
-| ScanInformation | scan_information | TEXT | **NEW**: Scan info |
-| Characters | tags | TEXT[] | Merged into tags |
-| Teams | tags | TEXT[] | Merged into tags |
-| Locations | tags | TEXT[] | Merged into tags |
+| Field | Database Column | Formats | Examples |
+| ------------------- | ------------------ | ------------ | ------------------------------------------------------------------------------------ |
+| age_rating | age_rating | All | Everyone, Teen, Mature, Adult |
+| series_count | series_count | All | "7" (Harry Potter), "12" (One Piece) |
+| volume | volume | All | "1" (Omnibus), "2" (Collected Edition) |
+| imprint | imprint | All | "Vertigo" (comics), "HarperCollinsEpic" (books) |
+| web_url | web_url | All | Goodreads, Audible, ComicVine, MangaUpdates URLs |
+| metadata_notes | metadata_notes | All | Publisher notes, scan info, EPUB annotations |
+| community_rating | community_rating | All | Pre-existing ratings from metadata sources |
+| Notes | metadata_notes | TEXT | **NEW**: Metadata notes (not user notes) |
+| Year | copyright_year | INTEGER | Already exists |
+| Month | (derived) | - | Stored in date_published |
+| Day | (derived) | - | Stored in date_published |
+| Writer | author | VARCHAR(255) | Already exists |
+| Penciller | contributors | TEXT[] | Merged into existing |
+| Inker | contributors | TEXT[] | Merged into existing |
+| Colorist | contributors | TEXT[] | Merged into existing |
+| Letterer | contributors | TEXT[] | Merged into existing |
+| CoverArtist | contributors | TEXT[] | Merged into existing |
+| Publisher | publisher | VARCHAR(255) | Already exists |
+| Imprint | imprint | VARCHAR(255) | **NEW**: Publisher imprint (UNIVERSAL) |
+| Genre | genre | VARCHAR(100) | Already exists + enhanced via processGenresAndTags() |
+| Tags | tags | TEXT[] | Already exists + enhanced with Characters/Teams/Locations via processGenresAndTags() |
+| Web | web_url | VARCHAR(500) | **NEW**: Info page URL (UNIVERSAL) |
+| PageCount | page_count | INTEGER | Already exists |
+| LanguageISO | language | VARCHAR(10) | Already exists |
+| Manga | manga_type | VARCHAR(30) | **NEW**: Raw manga field |
+| **Derived** | reading_direction | VARCHAR(20) | **NEW**: Computed from Manga field |
+| BlackAndWhite | is_black_and_white | BOOLEAN | **NEW**: B/W flag |
+| StoryArc | story_arc | VARCHAR(255) | **NEW**: Story arc name |
+| SeriesGroup | (future) | - | Not stored (can add later) |
+| AgeRating | age_rating | VARCHAR(20) | **NEW**: Age rating |
+| CommunityRating | community_rating | DOUBLE PRECISION | **NEW**: 0-10 rating |
+| MainCharacterOrTeam | tags | TEXT[] | Merged into tags |
+| Review | (future) | - | Not stored (user notes exist) |
+| ScanInformation | scan_information | TEXT | **NEW**: Scan info |
+| Characters | tags | TEXT[] | Merged into tags |
+| Teams | tags | TEXT[] | Merged into tags |
+| Locations | tags | TEXT[] | Merged into tags |
---
@@ -1932,13 +2327,15 @@ git push
Before considering this feature complete:
### Schema & Database
-- [ ] Database schema updated with 15 new columns (not 16 - removed genre_comic)
-- [ ] All 8 indexes created successfully (not 6 - added series_count and volume)
+
+- [ ] Database schema updated with 14 new columns (community_rating is DOUBLE PRECISION, not DECIMAL)
+- [ ] All 8 indexes created successfully (already in schema)
- [ ] Column comments added for all new fields
-- [ ] Database models regenerated with sqlc
-- [ ] Local database updated (Option 1, 2, or 3)
+- [ ] Database models regenerated with sqlc after schema change (CommunityRating becomes pgtype.Float8)
+- [ ] Local database updated (Option 1: recreate database, or Option 2: manual migration)
### Code Changes
+
- [ ] ComicInfo struct updated with all 19 fields
- [ ] MediaMetadata struct updated with 15 fields (5 universal + 10 comic-specific)
- [ ] mergeMetadata() function implements smart merging
@@ -1950,16 +2347,18 @@ Before considering this feature complete:
- [ ] determineReadingDirection() implements heuristics
- [ ] normalizeAgeRating() function standardizes ratings
- [ ] containsTag() helper prevents duplicate tags
-- [ ] processNewMediaItem() updated with all new fields
+- [ ] processMediaFile() updated with all new fields
- [ ] TypeScript types updated with all metadata fields
### Testing
+
- [ ] Unit tests pass (`go test ./internal/services/...`)
- [ ] Integration tests pass (`go test ./cmd/server/tests/...`)
- [ ] Bruno API tests pass
- [ ] Test coverage for all 19 ComicInfo.xml fields
### Manual Testing - Core Functionality
+
- [ ] Upload CBZ with `ComicInfo.xml` (Manga=YesAndRightToLeft)
- [ ] Verify RTL reading direction in database
- [ ] Upload CBZ with `metadata.opf` + `ComicInfo.xml`
@@ -1973,6 +2372,7 @@ Before considering this feature complete:
- [ ] Characters/Teams/Locations added to tags array
### Manual Testing - Specific Fields (Universal + Comic)
+
- [ ] `series_count` displays correctly in UI
- [ ] `volume` displays correctly in UI
- [ ] `imprint` displays and filters correctly (test with comics AND ebooks)
@@ -1987,6 +2387,7 @@ Before considering this feature complete:
- [ ] `summary` merges with description appropriately
### Manual Testing - Genre/Tag Processing
+
- [ ] EPUB subjects: first subject → genre, all subjects → tags
- [ ] ComicInfo Genre: → genre column
- [ ] ComicInfo Tags: → tags array
@@ -1997,12 +2398,14 @@ Before considering this feature complete:
- [ ] Primary genre set correctly (first genre tag wins)
### Documentation
+
- [ ] User documentation renders at `/docs` endpoint
- [ ] API documentation updated with all new fields
- [ ] Docs search finds new content
- [ ] Code examples in docs work
### Build & Verification
+
- [ ] `go build ./...` succeeds
- [ ] `bash scripts/verify-guidelines.sh` passes (0 errors)
- [ ] No critical functionality broken
@@ -2014,9 +2417,10 @@ Before considering this feature complete:
**Created:** March 29, 2026
**Updated:** March 29, 2026
**Changes:**
+
- v2.0: Added all 19 ComicInfo.xml fields (from 2 to 16 columns)
- v3.0: Made 5 fields universal (age_rating, series_count, volume, imprint, web_url)
- v3.0: Removed genre_comic column (use existing genre + processGenresAndTags)
- v3.0: Added genre/tag processing logic for ALL formats
- v3.0: Total: 15 new columns (5 universal + 10 comic-specific)
-**Status:** Ready for Implementation
+ **Status:** Ready for Implementation