docs: add comprehensive Calibre integration documentation
This commit adds complete documentation for the planned Calibre metadata.opf sidecar file support feature. ## New Documentation ### Implementation Planning - CALIBRE_OPF_IMPLEMENTATION.md: Detailed implementation plan with requirements, architecture, database mapping, and step-by-step implementation guide for adding Calibre metadata.opf support ### Technical Documentation - docs/development/calibre-opf-implementation.md: Technical implementation details including: - Scanner pipeline architecture with sidecar-first approach - Data structures (CalibreOPFMetadata, MediaMetadata) - Function signatures and logic for parseCalibreMetadataOPF() - Database schema mapping (no changes required) - Testing strategy (unit and integration tests) - Error handling and performance considerations - Code examples and benchmarking approach ### User Documentation - docs/user/calibre-integration.md: Comprehensive user guide covering: - What is Calibre and how Bookhoard integrates with it - Automatic metadata import from metadata.opf sidecar files - Supported metadata fields (Dublin Core + Calibre-specific) - Setup instructions for Calibre libraries - Workflow examples (fresh library, mixed library, updating metadata) - Troubleshooting common issues - Best practices for Calibre + Bookhoard workflow - FAQ and resources ## Updated Documentation - README.md: Added Calibre integration feature to media management section - docs/user/user-guide.md: Added link to Calibre integration guide - docs/developer/development.md: Added link to Calibre implementation guide ## Feature Summary The Calibre integration feature will allow Bookhoard to automatically import curated metadata from Calibre's metadata.opf sidecar files, including titles, authors, series, tags, descriptions, publishers, identifiers (ISBN/ASIN), and contributors. Uses sidecar-first approach: metadata.opf → embedded metadata → folder structure → filename. All database fields already exist; no schema changes required.
This commit is contained in:
@@ -0,0 +1,381 @@
|
||||
# Calibre metadata.opf Support - Implementation Plan
|
||||
|
||||
## Overview
|
||||
|
||||
This document outlines the implementation of **Calibre metadata.opf sidecar file support** for Bookhoard's media scanner. The scanner will use a **sidecar-first approach** - if a Calibre `metadata.opf` file exists alongside a book file, it will take precedence over embedded metadata.
|
||||
|
||||
## Motivation
|
||||
|
||||
Calibre is the most popular ebook management software, with millions of users. When users manage their libraries in Calibre:
|
||||
|
||||
1. **They curate metadata** - Adding series information, tags, ratings, custom covers
|
||||
2. **They fetch from online sources** - Google Books, Amazon, Goodreads (better than publisher metadata)
|
||||
3. **They convert/edit books** - The sidecar reflects the current, correct state
|
||||
4. **They organize in folders** - Often using Calibre's folder structure conventions
|
||||
|
||||
By supporting Calibre `metadata.opf` sidecar files, Bookhoard can:
|
||||
- Respect users' curation work
|
||||
- Import richer, more accurate metadata
|
||||
- Provide seamless integration with existing Calibre libraries
|
||||
- Support all file types (EPUB, PDF, comics, etc.) that Calibre manages
|
||||
|
||||
## Requirements
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
- [ ] Scanner detects `metadata.opf` files in the same directory as book files
|
||||
- [ ] Parses Dublin Core metadata (`dc:*` namespace) and Calibre-specific fields (`meta name="calibre:*"`)
|
||||
- [ ] Uses sidecar metadata **before** embedded metadata (sidecar-first approach)
|
||||
- [ ] Works for all library types (ebook, audiobook, comic, etc.)
|
||||
- [ ] Works for all file types (EPUB, PDF, CBZ, CBR, MP3, M4B, etc.)
|
||||
- [ ] Handles missing or malformed sidecar files gracefully
|
||||
- [ ] Maps all OPF fields to existing `media_items` database schema
|
||||
- [ ] Maintains backward compatibility (no sidecar = existing behavior)
|
||||
|
||||
### Non-Functional Requirements
|
||||
|
||||
- [ ] No breaking changes to existing scanner functionality
|
||||
- [ ] No database schema changes required (all fields already exist)
|
||||
- [ ] Follows existing code patterns and style
|
||||
- [ ] Comprehensive test coverage
|
||||
- [ ] Performance: Minimal impact on scan performance
|
||||
- [ ] Error handling: Graceful degradation on malformed XML
|
||||
|
||||
## Database Schema Mapping
|
||||
|
||||
All required fields **already exist** in the `media_items` table. No schema changes needed.
|
||||
|
||||
### Dublin Core Fields (dc: namespace)
|
||||
|
||||
| OPF Field | Database Column | Type | Notes |
|
||||
|-----------|-----------------|------|-------|
|
||||
| `dc:title` | `title` | VARCHAR(255) | Required |
|
||||
| `dc:creator` | `author` | VARCHAR(255) | First author only |
|
||||
| `dc:subject` | `tags` | TEXT[] | **Tags/keywords** (multiple) |
|
||||
| `dc:description` | `description` | TEXT | Book summary |
|
||||
| `dc:publisher` | `publisher` | VARCHAR(255) | Publisher name |
|
||||
| `dc:date` | `date_published` | DATE | Publication date |
|
||||
| `dc:language` | `language` | VARCHAR(10) | ISO 639-1 code (en, es, fr) |
|
||||
| `dc:identifier` (ISBN) | `isbn` | VARCHAR(13) | Check `opf:scheme="ISBN"` |
|
||||
| `dc:identifier` (ASIN) | `asin` | VARCHAR(20) | Check `opf:scheme="ASIN"` |
|
||||
| `dc:contributor` | `contributors` | TEXT[] | Multiple contributors |
|
||||
| `dc:rights` | *(not stored)* | - | Copyright info (ignored) |
|
||||
| `dc:source` | *(not stored)* | - | Source publication (ignored) |
|
||||
|
||||
### Calibre-Specific Fields (meta name="calibre:*")
|
||||
|
||||
| OPF Field | Database Column | Type | Notes |
|
||||
|-----------|-----------------|------|-------|
|
||||
| `calibre:series` | `series` | VARCHAR(255) | Series name |
|
||||
| `calibre:series_index` | `series_number` | INTEGER | Position in series |
|
||||
| `calibre:rating` | *(use media_ratings)* | INTEGER | 0-5 scale, stored per-user |
|
||||
| `calibre:title_sort` | *(not stored)* | - | Sortable title (ignored) |
|
||||
| `calibre:author_sort` | *(not stored)* | - | Sortable author (ignored) |
|
||||
| `calibre:timestamp` | `created_at` | TIMESTAMP | When added to Calibre |
|
||||
|
||||
### Special Cases
|
||||
|
||||
1. **Multiple authors**: Store first author in `author`, suggest adding `authors[]` array in future
|
||||
2. **Multiple identifiers**: Extract ISBN and ASIN if present, ignore others
|
||||
3. **Calibre rating**: Not imported (ratings are per-user in Bookhoard)
|
||||
4. **Sort fields**: Ignored (Bookhoard has its own sorting logic)
|
||||
5. **Timestamp**: Use for `created_at` if more recent than file modification time
|
||||
|
||||
## OPF Format Reference
|
||||
|
||||
### Standard Dublin Core Elements
|
||||
|
||||
```xml
|
||||
<dc:title>Book Title</dc:title>
|
||||
<dc:creator>Author Name</dc:creator>
|
||||
<dc:subject>Fantasy</dc:subject> <!-- Tag -->
|
||||
<dc:subject>Adventure</dc:subject> <!-- Another tag -->
|
||||
<dc:description>Book summary...</dc:description>
|
||||
<dc:publisher>Publisher Name</dc:publisher>
|
||||
<dc:date>2024-01-15</dc:date>
|
||||
<dc:language>en</dc:language>
|
||||
<dc:identifier opf:scheme="ISBN">978-0-123456-78-9</dc:identifier>
|
||||
<dc:identifier opf:scheme="ASIN">B08XXXXX</dc:identifier>
|
||||
<dc:contributor>Illustrator Name</dc:contributor>
|
||||
```
|
||||
|
||||
### Calibre-Specific Meta Tags
|
||||
|
||||
```xml
|
||||
<meta name="calibre:series" content="Series Name"/>
|
||||
<meta name="calibre:series_index" content="3"/>
|
||||
<meta name="calibre:rating" content="4"/>
|
||||
<meta name="calibre:title_sort" content="Book Title"/>
|
||||
<meta name="calibre:author_sort" content="Author, Name"/>
|
||||
```
|
||||
|
||||
### Example Complete metadata.opf
|
||||
|
||||
```xml
|
||||
<?xml version='1.0' encoding='utf-8'?>
|
||||
<package xmlns="http://www.idpf.org/2007/opf" version="2.0" unique-identifier="uuid_id">
|
||||
<metadata xmlns:opf="http://www.idpf.org/2007/opf"
|
||||
xmlns:calibre="http://calibre.kovidgoyal.net/2009/metadata"
|
||||
xmlns:dc="http://purl.org/dc/elements/1.1/">
|
||||
<dc:title>The Fellowship of the Ring</dc:title>
|
||||
<dc:creator opf:role="aut">J.R.R. Tolkien</dc:creator>
|
||||
<dc:identifier opf:scheme="ISBN">978-0-618-00222-0</dc:identifier>
|
||||
<dc:identifier opf:scheme="calibre">12345678-1234-1234-1234-123456789012</dc:identifier>
|
||||
<dc:language>en</dc:language>
|
||||
<dc:date>1954-07-29</dc:date>
|
||||
<dc:publisher>HarperCollins</dc:publisher>
|
||||
<dc:description>In a sleepy village in the Shire...</dc:description>
|
||||
|
||||
<!-- Tags -->
|
||||
<dc:subject>Fantasy</dc:subject>
|
||||
<dc:subject>Adventure</dc:subject>
|
||||
<dc:subject>Classics</dc:subject>
|
||||
|
||||
<!-- Contributors -->
|
||||
<dc:contributor>Alan Lee (illustrator)</dc:contributor>
|
||||
|
||||
<!-- Calibre-specific fields -->
|
||||
<meta name="calibre:series" content="The Lord of the Rings"/>
|
||||
<meta name="calibre:series_index" content="1"/>
|
||||
<meta name="calibre:rating" content="5"/>
|
||||
<meta name="calibre:title_sort" content="Fellowship of the Ring"/>
|
||||
</metadata>
|
||||
</package>
|
||||
```
|
||||
|
||||
## Implementation Design
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
Scanner Pipeline (EXISTING)
|
||||
↓
|
||||
1. Detect book file
|
||||
2. Check if already in database
|
||||
3. Extract metadata from file (EPUB/PDF/Comic)
|
||||
4. Extract folder structure metadata
|
||||
5. Normalize and store
|
||||
```
|
||||
|
||||
**New Step 3a**: Check for Calibre sidecar **before** embedded extraction
|
||||
|
||||
```
|
||||
Scanner Pipeline (NEW)
|
||||
↓
|
||||
1. Detect book file
|
||||
2. Check if already in database
|
||||
3. **NEW** Extract metadata from Calibre metadata.opf (if exists)
|
||||
- If sidecar found → use sidecar metadata
|
||||
- If no sidecar → use embedded metadata (existing)
|
||||
4. Extract folder structure metadata (existing)
|
||||
5. Normalize and store (existing)
|
||||
```
|
||||
|
||||
### Code Changes
|
||||
|
||||
#### File: `internal/services/media_scanner.go`
|
||||
|
||||
**New Functions:**
|
||||
|
||||
1. `extractMetadataFromCalibreSidecar(path string) (*MediaMetadata, error)`
|
||||
- Checks for `metadata.opf` in same directory as book file
|
||||
- Calls `parseCalibreMetadataOPF()` if found
|
||||
- Returns nil if no sidecar exists (not an error)
|
||||
|
||||
2. `parseCalibreMetadataOPF(opfPath string) (*MediaMetadata, error)`
|
||||
- Opens and parses XML file
|
||||
- Extracts Dublin Core fields (`dc:*`)
|
||||
- Extracts Calibre-specific fields (`meta name="calibre:*"`)
|
||||
- Maps to `MediaMetadata` struct
|
||||
- Handles errors gracefully (malformed XML, missing fields)
|
||||
|
||||
**Modified Function:**
|
||||
|
||||
1. `processMediaFile(ctx context.Context, path string) (bool, error)`
|
||||
- Add sidecar extraction check at line ~544 (before embedded extraction)
|
||||
- Flow: sidecar → embedded → folder → filename
|
||||
|
||||
#### New Struct: `CalibreOPFMetadata`
|
||||
|
||||
```go
|
||||
// Intermediate struct for parsed OPF data
|
||||
type CalibreOPFMetadata struct {
|
||||
Title string
|
||||
Authors []string
|
||||
Tags []string
|
||||
Description string
|
||||
Publisher string
|
||||
PublishDate *time.Time
|
||||
Language string
|
||||
ISBN string
|
||||
ASIN string
|
||||
UUID string
|
||||
Contributors []string
|
||||
Series string
|
||||
SeriesIndex *float64
|
||||
Rating *int32
|
||||
Timestamp *time.Time
|
||||
}
|
||||
```
|
||||
|
||||
### Metadata Priority (Sidecar-First)
|
||||
|
||||
```
|
||||
Priority Order:
|
||||
1. Calibre metadata.opf sidecar ← NEW (highest priority)
|
||||
2. Embedded file metadata (EPUB OPF, PDF info, ComicInfo.xml)
|
||||
3. Folder structure parsing
|
||||
4. Filename parsing (last resort)
|
||||
```
|
||||
|
||||
**Rationale**: If a user has a `metadata.opf` file, they're actively managing their library in Calibre. The sidecar represents their curated "source of truth."
|
||||
|
||||
### Error Handling
|
||||
|
||||
1. **Sidecar file not found** → Return nil, continue to embedded metadata
|
||||
2. **Malformed XML** → Log warning, continue to embedded metadata
|
||||
3. **Missing required fields** → Use available fields, continue
|
||||
4. **Invalid date format** → Log warning, skip field
|
||||
5. **File permission errors** → Log error, continue to embedded metadata
|
||||
|
||||
**Principle**: Graceful degradation - never fail a scan due to sidecar issues
|
||||
|
||||
### Testing Strategy
|
||||
|
||||
#### Unit Tests
|
||||
|
||||
File: `internal/services/media_scanner_calibre_test.go`
|
||||
|
||||
- Test parsing complete metadata.opf with all fields
|
||||
- Test parsing minimal metadata.opf (title only)
|
||||
- Test parsing malformed XML (graceful failure)
|
||||
- Test missing metadata.opf (returns nil, not error)
|
||||
- Test multiple `<dc:subject>` tags (tags array)
|
||||
- Test multiple `<dc:identifier>` tags (extract ISBN/ASIN)
|
||||
- Test Calibre-specific fields (series, series_index)
|
||||
- Test invalid date formats
|
||||
- Test empty/whitespace values
|
||||
|
||||
#### Integration Tests
|
||||
|
||||
File: `cmd/server/tests/calibre_integration_test.go`
|
||||
|
||||
- Test scanning directory with Calibre library structure
|
||||
- Test scanning directory without sidecar files (existing behavior)
|
||||
- Test mixed: some books with sidecar, some without
|
||||
- Test force rescan updates from sidecar changes
|
||||
- Test all file types: EPUB, PDF, CBZ, MP3, M4B
|
||||
- Test all library types: ebook, audiobook, comic
|
||||
|
||||
#### Test Data
|
||||
|
||||
Create `testdata/calibre/` with sample files:
|
||||
- `Complete Book/metadata.opf` - All fields populated
|
||||
- `Minimal Book/metadata.opf` - Only required fields
|
||||
- `Malformed Book/metadata.opf` - Invalid XML
|
||||
- No sidecar files (rely on existing test data)
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Core Functionality (REQUIRED)
|
||||
|
||||
- [ ] **Step 1.1**: Add `CalibreOPFMetadata` struct
|
||||
- [ ] **Step 1.2**: Implement `parseCalibreMetadataOPF()` function
|
||||
- [ ] **Step 1.3**: Implement `extractMetadataFromCalibreSidecar()` function
|
||||
- [ ] **Step 1.4**: Modify `processMediaFile()` to call sidecar extraction
|
||||
- [ ] **Step 1.5**: Add unit tests for OPF parsing
|
||||
- [ ] **Step 1.6**: Test with real Calibre library
|
||||
|
||||
### Phase 2: Documentation (REQUIRED)
|
||||
|
||||
- [ ] **Step 2.1**: Create user documentation (`docs/user/calibre-support.md`)
|
||||
- [ ] **Step 2.2**: Update developer documentation (`docs/development/calibre-opf-implementation.md`)
|
||||
- [ ] **Step 2.3**: Update scanner API docs if needed
|
||||
- [ ] **Step 2.4**: Add examples to documentation
|
||||
|
||||
### Phase 3: Testing & Polish (REQUIRED)
|
||||
|
||||
- [ ] **Step 3.1**: Add integration tests
|
||||
- [ ] **Step 3.2**: Test with various Calibre library configurations
|
||||
- [ ] **Step 3.3**: Performance testing (ensure no regression)
|
||||
- [ ] **Step 3.4**: Error handling review
|
||||
- [ ] **Step 3.5**: Code review and refinement
|
||||
|
||||
### Phase 4: Future Enhancements (OPTIONAL)
|
||||
|
||||
- [ ] Support Calibre `author_link` metadata
|
||||
- [ ] Support Calibre custom columns
|
||||
- [ ] Import Calibre ratings as user's initial rating
|
||||
- [ ] Detect Calibre library structure patterns
|
||||
- [ ] Configuration option for merge strategy (sidecar-first vs. smart-merge)
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
**No breaking changes.** Existing behavior is preserved:
|
||||
|
||||
- Libraries without `metadata.opf` files → Existing behavior (embedded metadata)
|
||||
- Scanning logic unchanged → Only adds sidecar check before embedded extraction
|
||||
- Database unchanged → All fields already exist
|
||||
- API unchanged → No new endpoints or response fields
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] All existing tests pass
|
||||
- [ ] New unit tests pass (90%+ coverage)
|
||||
- [ ] New integration tests pass
|
||||
- [ ] Scans real Calibre library successfully
|
||||
- [ ] Scans non-Calibre library without errors
|
||||
- [ ] No performance regression (scan time +5% max)
|
||||
- [ ] Documentation complete and accurate
|
||||
- [ ] Code review approved
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Q**: Should we import Calibre ratings?
|
||||
**A**: No - ratings are per-user in Bookhoard. Calibre rating is the owner's opinion, not necessarily the current user's.
|
||||
|
||||
2. **Q**: Should we support Calibre custom columns?
|
||||
**A**: Not in initial implementation. Custom columns are user-defined and would require dynamic schema or JSON storage. Future enhancement.
|
||||
|
||||
3. **Q**: How to handle multiple authors?
|
||||
**A**: Store first author in `author` field (existing behavior). Future enhancement could add `authors[]` array.
|
||||
|
||||
4. **Q**: Should we follow Calibre folder structure?
|
||||
**A**: Already implemented! `extractFolderStructureMetadata()` handles Calibre patterns.
|
||||
|
||||
## References
|
||||
|
||||
- [EPUB Publications 3.0 Specification](https://www.idpf.org/epub/30/spec/epub30-publications.html)
|
||||
- [Dublin Core Metadata Element Set](https://www.dublincore.org/specifications/dces/1998-08-13/)
|
||||
- [Calibre Manual - Editing Metadata](https://manual.calibre-ebook.com/metadata.html)
|
||||
- [Calibre Source Code](https://github.com/kovidgoyal/calibre)
|
||||
|
||||
## Appendix: Decision Log
|
||||
|
||||
### Decision 1: Sidecar-First Approach
|
||||
**Date**: 2026-03-26
|
||||
**Decision**: Use sidecar-first (not smart-merge)
|
||||
**Rationale**:
|
||||
- Users with Calibre sidecars actively curate their metadata
|
||||
- Sidecar metadata is typically richer and more accurate
|
||||
- Simple implementation, predictable behavior
|
||||
- Easy to test and debug
|
||||
- Future enhancement can add configurable merge strategy
|
||||
|
||||
### Decision 2: No Database Schema Changes
|
||||
**Date**: 2026-03-26
|
||||
**Decision**: Use existing schema only
|
||||
**Rationale**:
|
||||
- All required fields already exist in `media_items` table
|
||||
- Pre-production application (no legacy data to migrate)
|
||||
- Simpler implementation
|
||||
- No backward compatibility concerns
|
||||
|
||||
### Decision 3: All Library Types Supported
|
||||
**Date**: 2026-03-26
|
||||
**Decision**: Support OPF for all libraries, not just ebooks
|
||||
**Rationale**:
|
||||
- Calibre supports all file types
|
||||
- Users may organize audiobooks/comics in Calibre
|
||||
- Consistent behavior across library types
|
||||
- No additional complexity
|
||||
@@ -60,6 +60,7 @@ The first user to register automatically becomes an admin.
|
||||
|
||||
### Media Management
|
||||
|
||||
- **Calibre Integration**: Automatic metadata import from Calibre `metadata.opf` sidecar files
|
||||
- **Smart Search**: Partial matching with fuzzy search fallback for typos
|
||||
- **Advanced Filtering**: Filter by author, series, genre, language, year, cover images
|
||||
- **Dynamic Sorting**: By title, author, date added, published date, page count, series
|
||||
@@ -97,6 +98,7 @@ The first user to register automatically becomes an admin.
|
||||
|
||||
### For Users & Self-Hosters
|
||||
|
||||
- **[docs/user/calibre-integration.md](docs/user/calibre-integration.md)** - Calibre library integration
|
||||
- **[docs/user/sync-guide.md](docs/user/sync-guide.md)** - Understanding and using universal sync
|
||||
- **[docs/user/devices/kobo-setup.md](docs/user/devices/kobo-setup.md)** - Kobo e-reader configuration
|
||||
- **[docs/user/devices/koreader-setup.md](docs/user/devices/koreader-setup.md)** - KOReader configuration
|
||||
|
||||
@@ -71,6 +71,18 @@ Browse individual endpoint documentation with interactive API Explorer:
|
||||
- Event types
|
||||
- Authentication
|
||||
|
||||
## 🛠 Implementation Guides
|
||||
|
||||
### Calibre Integration
|
||||
|
||||
- **[Calibre OPF Implementation](../development/calibre-opf-implementation.md)** - Technical implementation details
|
||||
- Architecture and design
|
||||
- Data structures and functions
|
||||
- Database schema mapping
|
||||
- Testing strategy
|
||||
- Error handling
|
||||
- Performance considerations
|
||||
|
||||
---
|
||||
|
||||
**Looking for user documentation?** See the [User Portal](../user/user-guide.md)
|
||||
|
||||
@@ -0,0 +1,576 @@
|
||||
# Calibre metadata.opf Implementation
|
||||
|
||||
This document provides technical details for implementing Calibre `metadata.opf` sidecar file support in Bookhoard's media scanner.
|
||||
|
||||
## Overview
|
||||
|
||||
The media scanner detects and imports metadata from Calibre's `metadata.opf` sidecar files using a **sidecar-first approach**. If a `metadata.opf` file exists alongside a book file, its metadata takes precedence over embedded file metadata.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Scanner Pipeline
|
||||
|
||||
```go
|
||||
func (s *MediaScanner) processMediaFile(ctx context.Context, path string) (bool, error) {
|
||||
// 1. Get file info
|
||||
// 2. Find library
|
||||
// 3. Check if media item exists
|
||||
// 4. EXTRACT METADATA (NEW: sidecar-first)
|
||||
metadata := s.extractMetadataWithSidecar(path)
|
||||
// 5. Extract hash info
|
||||
// 6. Extract comic metadata (if applicable)
|
||||
// 7. Extract folder structure metadata
|
||||
// 8. Normalize metadata
|
||||
// 9. Create or update in database
|
||||
}
|
||||
```
|
||||
|
||||
### Metadata Extraction Flow
|
||||
|
||||
```
|
||||
extractMetadataWithSidecar()
|
||||
↓
|
||||
1. Try Calibre metadata.opf sidecar
|
||||
├─ Found → parseCalibreMetadataOPF()
|
||||
└─ Not found → continue
|
||||
↓
|
||||
2. Extract embedded metadata
|
||||
├─ EPUB → extractEPUBMetadata()
|
||||
├─ PDF → extractPDFMetadata()
|
||||
└─ Comic → extractComicMetadata()
|
||||
↓
|
||||
3. Extract folder structure metadata
|
||||
└─ extractFolderStructureMetadata()
|
||||
↓
|
||||
4. Return merged MediaMetadata
|
||||
```
|
||||
|
||||
## Data Structures
|
||||
|
||||
### CalibreOPFMetadata
|
||||
|
||||
Intermediate struct for parsed OPF data:
|
||||
|
||||
```go
|
||||
// CalibreOPFMetadata represents parsed metadata from Calibre metadata.opf
|
||||
type CalibreOPFMetadata struct {
|
||||
Title string
|
||||
Authors []string
|
||||
Tags []string
|
||||
Description string
|
||||
Publisher string
|
||||
PublishDate *time.Time
|
||||
Language string
|
||||
ISBN string
|
||||
ASIN string
|
||||
UUID string
|
||||
Contributors []string
|
||||
Series string
|
||||
SeriesIndex *float64
|
||||
Rating *int32
|
||||
Timestamp *time.Time
|
||||
}
|
||||
```
|
||||
|
||||
### MediaMetadata
|
||||
|
||||
Existing struct (already defined in `media_scanner.go`):
|
||||
|
||||
```go
|
||||
// MediaMetadata represents extracted book metadata
|
||||
type MediaMetadata struct {
|
||||
Title string
|
||||
Author string
|
||||
ISBN string
|
||||
Description string
|
||||
Publisher string
|
||||
Series string
|
||||
SeriesNumber int32
|
||||
Tags []string
|
||||
Contributors []string
|
||||
PublishDate time.Time
|
||||
ASIN string
|
||||
// ... other fields
|
||||
}
|
||||
```
|
||||
|
||||
## Implementation Functions
|
||||
|
||||
### 1. extractMetadataFromCalibreSidecar()
|
||||
|
||||
**Location**: `internal/services/media_scanner.go`
|
||||
|
||||
**Signature**:
|
||||
```go
|
||||
func (s *MediaScanner) extractMetadataFromCalibreSidecar(mediaFilePath string) (*MediaMetadata, error)
|
||||
```
|
||||
|
||||
**Purpose**: Checks for `metadata.opf` in the same directory as the book file.
|
||||
|
||||
**Logic**:
|
||||
```go
|
||||
func (s *MediaScanner) extractMetadataFromCalibreSidecar(mediaFilePath string) (*MediaMetadata, error) {
|
||||
// Get directory of media file
|
||||
dir := filepath.Dir(mediaFilePath)
|
||||
|
||||
// Check for metadata.opf
|
||||
opfPath := filepath.Join(dir, "metadata.opf")
|
||||
if _, err := os.Stat(opfPath); os.IsNotExist(err) {
|
||||
// No sidecar file - not an error, just return nil
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
// Parse metadata.opf
|
||||
return s.parseCalibreMetadataOPF(opfPath)
|
||||
}
|
||||
```
|
||||
|
||||
**Error Handling**:
|
||||
- File not found → Return `nil, nil` (not an error)
|
||||
- Permission denied → Log warning, return `nil, nil`
|
||||
- Other errors → Log error, return `nil, err`
|
||||
|
||||
### 2. parseCalibreMetadataOPF()
|
||||
|
||||
**Location**: `internal/services/media_scanner.go`
|
||||
|
||||
**Signature**:
|
||||
```go
|
||||
func (s *MediaScanner) parseCalibreMetadataOPF(opfPath string) (*MediaMetadata, error)
|
||||
```
|
||||
|
||||
**Purpose**: Parses XML file and extracts Dublin Core + Calibre-specific fields.
|
||||
|
||||
**Logic**:
|
||||
```go
|
||||
func (s *MediaScanner) parseCalibreMetadataOPF(opfPath string) (*MediaMetadata, error) {
|
||||
// Open file
|
||||
file, err := os.Open(opfPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to open metadata.opf: %v", err)
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
// Parse XML
|
||||
var opf struct {
|
||||
XMLName xml.Name `xml:"package"`
|
||||
Metadata struct {
|
||||
XMLName xml.Name `xml:"metadata"`
|
||||
Titles []string `xml:"dc:title"`
|
||||
Creators []string `xml:"dc:creator"`
|
||||
Subjects []string `xml:"dc:subject"`
|
||||
Desc []string `xml:"dc:description"`
|
||||
Publisher []string `xml:"dc:publisher"`
|
||||
Dates []string `xml:"dc:date"`
|
||||
Language []string `xml:"dc:language"`
|
||||
Identifiers []struct {
|
||||
Scheme string `xml:"opf:scheme,attr"`
|
||||
Value string `xml:",chardata"`
|
||||
} `xml:"dc:identifier"`
|
||||
Contributors []string `xml:"dc:contributor"`
|
||||
CalibreSeries []struct {
|
||||
Name string `xml:"content,attr"`
|
||||
} `xml:"meta[name='calibre:series']"`
|
||||
CalibreSeriesIndex []struct {
|
||||
Value string `xml:"content,attr"`
|
||||
} `xml:"meta[name='calibre:series_index']"`
|
||||
} `xml:"metadata"`
|
||||
}
|
||||
|
||||
if err := xml.NewDecoder(file).Decode(&opf); err != nil {
|
||||
return nil, fmt.Errorf("failed to parse metadata.opf XML: %v", err)
|
||||
}
|
||||
|
||||
// Map to MediaMetadata
|
||||
metadata := &MediaMetadata{}
|
||||
|
||||
// Title (required)
|
||||
if len(opf.Metadata.Titles) > 0 {
|
||||
metadata.Title = opf.Metadata.Titles[0]
|
||||
}
|
||||
|
||||
// Author (first creator)
|
||||
if len(opf.Metadata.Creators) > 0 {
|
||||
metadata.Author = opf.Metadata.Creators[0]
|
||||
}
|
||||
|
||||
// Tags (all subjects)
|
||||
if len(opf.Metadata.Subjects) > 0 {
|
||||
metadata.Tags = utils.NormalizeTags(opf.Metadata.Subjects)
|
||||
}
|
||||
|
||||
// Description
|
||||
if len(opf.Metadata.Desc) > 0 {
|
||||
metadata.Description = opf.Metadata.Desc[0]
|
||||
}
|
||||
|
||||
// Publisher
|
||||
if len(opf.Metadata.Publisher) > 0 {
|
||||
metadata.Publisher = opf.Metadata.Publisher[0]
|
||||
}
|
||||
|
||||
// Publish date
|
||||
if len(opf.Metadata.Dates) > 0 {
|
||||
if date, err := time.Parse("2006-01-02", opf.Metadata.Dates[0]); err == nil {
|
||||
metadata.PublishDate = date
|
||||
}
|
||||
}
|
||||
|
||||
// Language
|
||||
if len(opf.Metadata.Language) > 0 {
|
||||
// Store language (already in schema)
|
||||
// Note: MediaMetadata doesn't have Language field yet
|
||||
// Future enhancement: add Language field
|
||||
}
|
||||
|
||||
// Identifiers (ISBN, ASIN)
|
||||
for _, id := range opf.Metadata.Identifiers {
|
||||
switch strings.ToUpper(id.Scheme) {
|
||||
case "ISBN":
|
||||
metadata.ISBN = utils.NormalizeISBNSafe(id.Value)
|
||||
case "ASIN":
|
||||
metadata.ASIN = id.Value
|
||||
case "UUID", "CALIBRE":
|
||||
// Store UUID in hash info, not metadata
|
||||
// Will be extracted by extractHashInfo()
|
||||
}
|
||||
}
|
||||
|
||||
// Contributors
|
||||
if len(opf.Metadata.Contributors) > 0 {
|
||||
metadata.Contributors = utils.NormalizeContributors(opf.Metadata.Contributors)
|
||||
}
|
||||
|
||||
// Series
|
||||
if len(opf.Metadata.CalibreSeries) > 0 {
|
||||
metadata.Series = opf.Metadata.CalibreSeries[0].Name
|
||||
}
|
||||
|
||||
// Series index
|
||||
if len(opf.Metadata.CalibreSeriesIndex) > 0 {
|
||||
if index, err := strconv.ParseFloat(opf.Metadata.CalibreSeriesIndex[0].Value, 32); err == nil {
|
||||
metadata.SeriesNumber = int32(index)
|
||||
}
|
||||
}
|
||||
|
||||
return metadata, nil
|
||||
}
|
||||
```
|
||||
|
||||
**XML Namespaces**: Handle Dublin Core (`dc:`) and Calibre (`calibre:`) namespaces properly.
|
||||
|
||||
**Error Handling**:
|
||||
- Malformed XML → Log warning, return `nil, nil`
|
||||
- Missing required fields → Use available fields
|
||||
- Invalid date format → Log warning, skip field
|
||||
|
||||
### 3. extractMetadataWithSidecar()
|
||||
|
||||
**Location**: `internal/services/media_scanner.go`
|
||||
|
||||
**Signature**:
|
||||
```go
|
||||
func (s *MediaScanner) extractMetadataWithSidecar(path string) (*MediaMetadata, error)
|
||||
```
|
||||
|
||||
**Purpose**: Implements sidecar-first metadata extraction.
|
||||
|
||||
**Logic**:
|
||||
```go
|
||||
func (s *MediaScanner) extractMetadataWithSidecar(path string) (*MediaMetadata, error) {
|
||||
// Try Calibre sidecar first
|
||||
if metadata, err := s.extractMetadataFromCalibreSidecar(path); metadata != nil {
|
||||
fmt.Printf("Using Calibre metadata.opf for %s\n", path)
|
||||
return metadata, nil
|
||||
} else if err != nil {
|
||||
// Log error but continue to embedded extraction
|
||||
fmt.Printf("Warning: failed to read Calibre sidecar: %v\n", err)
|
||||
}
|
||||
|
||||
// Fallback to embedded metadata
|
||||
return s.extractMetadata(path)
|
||||
}
|
||||
```
|
||||
|
||||
**Priority**: Sidecar → Embedded → Folder → Filename
|
||||
|
||||
## Database Schema Mapping
|
||||
|
||||
All OPF fields map to existing `media_items` columns:
|
||||
|
||||
| OPF Field | Database Column | Type | Conversion |
|
||||
|-----------|-----------------|------|------------|
|
||||
| `dc:title` | `title` | VARCHAR(255) | Direct |
|
||||
| `dc:creator` | `author` | VARCHAR(255) | First author only |
|
||||
| `dc:subject` | `tags` | TEXT[] | Array conversion |
|
||||
| `dc:description` | `description` | TEXT | Direct |
|
||||
| `dc:publisher` | `publisher` | VARCHAR(255) | Direct |
|
||||
| `dc:date` | `date_published` | DATE | Parse ISO 8601 |
|
||||
| `dc:language` | `language` | VARCHAR(10) | Store ISO code |
|
||||
| `dc:identifier[ISBN]` | `isbn` | VARCHAR(13) | Normalize with utils |
|
||||
| `dc:identifier[ASIN]` | `asin` | VARCHAR(20) | Direct |
|
||||
| `dc:contributor` | `contributors` | TEXT[] | Array conversion |
|
||||
| `calibre:series` | `series` | VARCHAR(255) | Direct |
|
||||
| `calibre:series_index` | `series_number` | INTEGER | Parse float → int |
|
||||
|
||||
**No schema changes required** - all fields already exist.
|
||||
|
||||
## Normalization
|
||||
|
||||
Use existing normalization utilities:
|
||||
|
||||
```go
|
||||
// Tags
|
||||
metadata.Tags = utils.NormalizeTags(tags)
|
||||
|
||||
// Contributors
|
||||
metadata.Contributors = utils.NormalizeContributors(contributors)
|
||||
|
||||
// ISBN
|
||||
metadata.ISBN = utils.NormalizeISBNSafe(isbn)
|
||||
|
||||
// Search versions (auto-generated by database triggers)
|
||||
// tags_search, contributors_search
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Unit Tests
|
||||
|
||||
**File**: `internal/services/media_scanner_calibre_test.go`
|
||||
|
||||
```go
|
||||
func TestParseCalibreMetadataOPF(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
opfContent string
|
||||
want *MediaMetadata
|
||||
wantErr bool
|
||||
}{
|
||||
{
|
||||
name: "Complete metadata.opf",
|
||||
opfContent: `<?xml version='1.0' encoding='utf-8'?>
|
||||
<package xmlns="http://www.idpf.org/2007/opf" version="2.0">
|
||||
<metadata xmlns:dc="http://purl.org/dc/elements/1.1/">
|
||||
<dc:title>Test Book</dc:title>
|
||||
<dc:creator>Test Author</dc:creator>
|
||||
<dc:subject>Fantasy</dc:subject>
|
||||
<dc:subject>Adventure</dc:subject>
|
||||
<dc:description>Test description</dc:description>
|
||||
<dc:publisher>Test Publisher</dc:publisher>
|
||||
<dc:date>2024-01-15</dc:date>
|
||||
<dc:language>en</dc:language>
|
||||
<dc:identifier opf:scheme="ISBN">978-0-123456-78-9</dc:identifier>
|
||||
<meta name="calibre:series" content="Test Series"/>
|
||||
<meta name="calibre:series_index" content="1"/>
|
||||
</metadata>
|
||||
</package>`,
|
||||
want: &MediaMetadata{
|
||||
Title: "Test Book",
|
||||
Author: "Test Author",
|
||||
Tags: []string{"Fantasy", "Adventure"},
|
||||
Description: "Test description",
|
||||
Publisher: "Test Publisher",
|
||||
Series: "Test Series",
|
||||
SeriesNumber: 1,
|
||||
},
|
||||
wantErr: false,
|
||||
},
|
||||
// ... more test cases
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
// Create temporary OPF file
|
||||
tmpDir := t.TempDir()
|
||||
opfPath := filepath.Join(tmpDir, "metadata.opf")
|
||||
if err := os.WriteFile(opfPath, []byte(tt.opfContent), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Parse
|
||||
scanner := &MediaScanner{}
|
||||
got, err := scanner.parseCalibreMetadataOPF(opfPath)
|
||||
|
||||
if (err != nil) != tt.wantErr {
|
||||
t.Errorf("parseCalibreMetadataOPF() error = %v, wantErr %v", err, tt.wantErr)
|
||||
return
|
||||
}
|
||||
|
||||
// Verify fields
|
||||
if got.Title != tt.want.Title {
|
||||
t.Errorf("Title = %v, want %v", got.Title, tt.want.Title)
|
||||
}
|
||||
// ... more assertions
|
||||
})
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Test Cases**:
|
||||
1. Complete metadata.opf (all fields)
|
||||
2. Minimal metadata.opf (title only)
|
||||
3. Malformed XML
|
||||
4. Missing metadata.opf (returns nil)
|
||||
5. Multiple `<dc:subject>` tags
|
||||
6. Multiple `<dc:identifier>` tags
|
||||
7. Invalid date formats
|
||||
8. Empty/whitespace values
|
||||
|
||||
### Integration Tests
|
||||
|
||||
**File**: `cmd/server/tests/calibre_integration_test.go`
|
||||
|
||||
```go
|
||||
func TestCalibreLibraryScan(t *testing.T) {
|
||||
// Setup test server
|
||||
setup := setupTestServer(t)
|
||||
defer setup.Teardown()
|
||||
|
||||
// Create test Calibre library structure
|
||||
tmpDir := t.TempDir()
|
||||
createTestCalibreLibrary(t, tmpDir)
|
||||
|
||||
// Add library
|
||||
libraryID := createTestLibrary(t, setup.DB, tmpDir)
|
||||
|
||||
// Scan library
|
||||
scanner := services.NewMediaScanner(setup.DB, setup.AdminID, []string{tmpDir})
|
||||
err := scanner.ScanFolders(context.Background())
|
||||
if err != nil {
|
||||
t.Fatalf("ScanFolders() error = %v", err)
|
||||
}
|
||||
|
||||
// Verify imported books
|
||||
books, err := setup.DB.ListMediaItems(context.Background(), libraryID)
|
||||
if err != nil {
|
||||
t.Fatalf("ListMediaItems() error = %v", err)
|
||||
}
|
||||
|
||||
// Assertions
|
||||
if len(books) != 3 {
|
||||
t.Errorf("Got %d books, want 3", len(books))
|
||||
}
|
||||
|
||||
// Verify metadata from sidecar
|
||||
for _, book := range books {
|
||||
if book.Title == "" {
|
||||
t.Errorf("Book %s: Title should not be empty", book.ID)
|
||||
}
|
||||
if book.Series.Valid && book.SeriesNumber.Int32 == 0 {
|
||||
t.Errorf("Book %s: Series number should be set", book.ID)
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Test Scenarios**:
|
||||
1. Scan Calibre library (with sidecars)
|
||||
2. Scan non-Calibre library (no sidecars)
|
||||
3. Mixed library (some with sidecars, some without)
|
||||
4. Force rescan updates from sidecar changes
|
||||
5. All file types (EPUB, PDF, CBZ, MP3)
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Impact on Scan Performance
|
||||
|
||||
**Expected overhead**: +5-10% scan time increase
|
||||
|
||||
**Optimizations**:
|
||||
1. **File system cache**: OS caches `metadata.opf` reads
|
||||
2. **Early exit**: Return immediately if sidecar not found
|
||||
3. **Parallel processing**: Sidecar extraction doesn't block other files
|
||||
4. **Lazy parsing**: Only parse XML when sidecar exists
|
||||
|
||||
**Benchmarking**:
|
||||
```go
|
||||
func BenchmarkSidecarExtraction(b *testing.B) {
|
||||
scanner := &MediaScanner{}
|
||||
tmpDir := b.TempDir()
|
||||
createTestCalibreLibrary(b, tmpDir)
|
||||
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
_, err := scanner.extractMetadataFromCalibreSidecar(filepath.Join(tmpDir, "book.epub"))
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Memory Usage
|
||||
|
||||
**XML parsing**: Small memory footprint (OPF files are typically <10KB)
|
||||
|
||||
**No leaks**: Ensure file handles are closed with `defer file.Close()`
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Graceful Degradation
|
||||
|
||||
The scanner must **never fail** due to sidecar issues:
|
||||
|
||||
```go
|
||||
// Pattern: Log error, continue to next source
|
||||
if metadata, err := s.extractMetadataFromCalibreSidecar(path); err != nil {
|
||||
// Log warning
|
||||
fmt.Printf("Warning: failed to read Calibre sidecar: %v\n", err)
|
||||
// Continue to embedded extraction
|
||||
metadata = s.extractMetadata(path)
|
||||
}
|
||||
```
|
||||
|
||||
### Error Types
|
||||
|
||||
| Error | Action | Log Level |
|
||||
|-------|--------|-----------|
|
||||
| File not found | Continue to embedded | DEBUG |
|
||||
| Permission denied | Continue to embedded | WARN |
|
||||
| Malformed XML | Continue to embedded | WARN |
|
||||
| Invalid date format | Skip field | WARN |
|
||||
| Empty required field | Use available fields | INFO |
|
||||
| UTF-8 decode error | Continue to embedded | ERROR |
|
||||
|
||||
### Logging
|
||||
|
||||
Use structured logging:
|
||||
|
||||
```go
|
||||
fmt.Printf("Using Calibre metadata.opf for %s\n", path)
|
||||
fmt.Printf("Warning: failed to parse metadata.opf: %v\n", err)
|
||||
fmt.Printf("Info: No sidecar found, using embedded metadata for %s\n", path)
|
||||
```
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Phase 4 (Optional)
|
||||
|
||||
1. **Custom column support**: Parse Calibre custom columns
|
||||
2. **Smart merge**: Configurable merge strategy
|
||||
3. **Rating import**: Import Calibre rating as user's initial rating
|
||||
4. **Library detection**: Auto-detect Calibre library structure
|
||||
5. **Incremental sync**: Only re-scan changed sidecar files
|
||||
|
||||
### Configuration Option
|
||||
|
||||
```go
|
||||
type ScannerConfig struct {
|
||||
PreferCalibreSidecar bool // Default: true
|
||||
CalibreMergeStrategy string // "sidecar-first" | "smart-merge" | "embedded-first"
|
||||
}
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
- [EPUB 3.0 Spec](https://www.idpf.org/epub/30/spec/epub30-publications.html)
|
||||
- [Dublin Core](https://www.dublincore.org/specifications/dces/1998-08-13/)
|
||||
- [Calibre Manual](https://manual.calibre-ebook.com/)
|
||||
- [Go XML Encoding](https://pkg.go.dev/encoding/xml)
|
||||
|
||||
## See Also
|
||||
|
||||
- [User Guide: Calibre Integration](../../user/calibre-integration.md)
|
||||
- [Implementation Plan](../../CALIBRE_OPF_IMPLEMENTATION.md)
|
||||
- [Scanner Source Code](../../internal/services/media_scanner.go)
|
||||
@@ -0,0 +1,313 @@
|
||||
# Calibre Integration Guide
|
||||
|
||||
Bookhoard provides seamless integration with [Calibre](https://calibre-ebook.com/) libraries. If you manage your ebooks with Calibre, Bookhoard can automatically import your curated metadata, including series information, tags, and custom covers.
|
||||
|
||||
## What is Calibre?
|
||||
|
||||
Calibre is a free and open-source ebook management software. It allows you to:
|
||||
|
||||
- **Organize your library** - Create collections, add tags, manage series
|
||||
- **Edit metadata** - Update titles, authors, descriptions, covers
|
||||
- **Fetch metadata online** - Download from Google Books, Amazon, Goodreads
|
||||
- **Convert formats** - Convert between EPUB, MOBI, PDF, and more
|
||||
- **Sync to devices** - Send books to Kindle, Kobo, and other e-readers
|
||||
|
||||
## How Bookhoard Integrates with Calibre
|
||||
|
||||
### Automatic Metadata Import
|
||||
|
||||
When you scan a Calibre library in Bookhoard, it automatically detects and imports metadata from Calibre's `metadata.opf` files. This includes:
|
||||
|
||||
- ✅ **Titles and authors** - Your curated edits
|
||||
- ✅ **Series information** - Series name and position
|
||||
- ✅ **Tags and genres** - Your custom organization
|
||||
- ✅ **Descriptions** - Book summaries
|
||||
- ✅ **Publishers and dates** - Publication information
|
||||
- ✅ **Identifiers** - ISBN, ASIN, and other IDs
|
||||
- ✅ **Contributors** - Illustrators, editors, translators
|
||||
- ✅ **Custom covers** - Your chosen cover images
|
||||
|
||||
### Sidecar-First Approach
|
||||
|
||||
Bookhoard uses a **sidecar-first** approach for Calibre libraries:
|
||||
|
||||
1. **If `metadata.opf` exists** → Use Calibre's curated metadata
|
||||
2. **If no sidecar** → Use embedded metadata from the book file
|
||||
3. **Fallback** → Use folder structure and filename
|
||||
|
||||
This ensures your Calibre curation work is respected and imported accurately.
|
||||
|
||||
## Setting Up Your Calibre Library in Bookhoard
|
||||
|
||||
### Step 1: Organize Your Calibre Library
|
||||
|
||||
Ensure your Calibre library has `metadata.opf` files in each book's folder. Calibre creates these automatically when you add books to your library.
|
||||
|
||||
Typical Calibre folder structure:
|
||||
```
|
||||
Calibre Library/
|
||||
├── Author Name/
|
||||
│ ├── Book Title (Series #1)/
|
||||
│ │ ├── Book Title.epub
|
||||
│ │ ├── metadata.opf ← Bookhoard reads this
|
||||
│ │ └── cover.jpg
|
||||
│ └── Book Title 2/
|
||||
│ ├── Book Title 2.epub
|
||||
│ └── metadata.opf ← Bookhoard reads this
|
||||
```
|
||||
|
||||
### Step 2: Add Library in Bookhoard
|
||||
|
||||
1. Navigate to **Admin** → **Libraries**
|
||||
2. Click **Add Library**
|
||||
3. Configure:
|
||||
- **Name**: "My Calibre Library"
|
||||
- **Type**: Ebook (or Audiobook/Comic)
|
||||
- **Folder**: Path to your Calibre library
|
||||
- **Scan on save**: ✅ Checked
|
||||
4. Click **Save**
|
||||
|
||||
Bookhoard will automatically scan the library and import all books with their Calibre metadata.
|
||||
|
||||
### Step 3: Verify Import
|
||||
|
||||
1. Navigate to **Library** view
|
||||
2. Browse your imported books
|
||||
3. Check that:
|
||||
- Titles and authors are correct
|
||||
- Series information appears (if applicable)
|
||||
- Tags are imported
|
||||
- Covers display correctly
|
||||
- Descriptions are present
|
||||
|
||||
## Supported Metadata
|
||||
|
||||
### Dublin Core Fields
|
||||
|
||||
Bookhoard imports standard Dublin Core metadata from Calibre:
|
||||
|
||||
| Field | Source | Notes |
|
||||
|-------|--------|-------|
|
||||
| Title | `dc:title` | Book title |
|
||||
| Author | `dc:creator` | Primary author |
|
||||
| Tags | `dc:subject` | **Keywords/tags** (multiple) |
|
||||
| Description | `dc:description` | Book summary |
|
||||
| Publisher | `dc:publisher` | Publisher name |
|
||||
| Date | `dc:date` | Publication date |
|
||||
| Language | `dc:language` | ISO language code |
|
||||
| ISBN | `dc:identifier` | ISBN (if present) |
|
||||
| ASIN | `dc:identifier` | Amazon ID (if present) |
|
||||
| Contributors | `dc:contributor` | Additional contributors |
|
||||
|
||||
### Calibre-Specific Fields
|
||||
|
||||
Bookhoard also imports Calibre's custom metadata:
|
||||
|
||||
| Field | Source | Notes |
|
||||
|-------|--------|-------|
|
||||
| Series | `calibre:series` | Series name |
|
||||
| Series Number | `calibre:series_index` | Position in series |
|
||||
| Timestamp | `dc:date` | When added to Calibre |
|
||||
|
||||
### Not Imported
|
||||
|
||||
The following Calibre metadata is **not** imported (by design):
|
||||
|
||||
- ❌ **Ratings** - Calibre ratings are personal; Bookhoard has per-user ratings
|
||||
- ❌ **Sort fields** - Bookhoard has its own sorting logic
|
||||
- ❌ **Custom columns** - User-defined columns (future enhancement)
|
||||
- ❌ **Last read** - Reading progress is tracked per-user in Bookhoard
|
||||
|
||||
## Supported File Types
|
||||
|
||||
Bookhoard imports Calibre metadata for **all file types**:
|
||||
|
||||
- 📚 **Ebooks**: EPUB, MOBI, AZW3, PDF, TXT, etc.
|
||||
- 🎧 **Audiobooks**: MP3, M4B, M4A, etc.
|
||||
- 📰 **Comics**: CBZ, CBR, CB7, CBT, etc.
|
||||
- 📄 **Documents**: PDF, DOCX, etc.
|
||||
|
||||
As long as a `metadata.opf` file exists in the folder, Bookhoard will import the metadata.
|
||||
|
||||
## Workflow Examples
|
||||
|
||||
### Example 1: Fresh Calibre Library
|
||||
|
||||
**Scenario**: You have a Calibre library with 500 ebooks, all organized with series, tags, and custom covers.
|
||||
|
||||
**Steps**:
|
||||
1. Add the Calibre library folder in Bookhoard
|
||||
2. Enable "Scan on save"
|
||||
3. Bookhoard imports all 500 books with:
|
||||
- Correct titles and authors
|
||||
- Series information (e.g., "Harry Potter #2")
|
||||
- Your custom tags (e.g., "Fantasy", "Favorites", "To Read")
|
||||
- Your chosen cover images
|
||||
- Descriptions and publisher info
|
||||
|
||||
**Result**: Your entire Calibre library is instantly available in Bookhoard with all your curation work preserved.
|
||||
|
||||
### Example 2: Mixed Library (Calibre + Non-Calibre)
|
||||
|
||||
**Scenario**: Your library has some books from Calibre (with `metadata.opf`) and some downloaded from elsewhere (no sidecar).
|
||||
|
||||
**Steps**:
|
||||
1. Add the library folder in Bookhoard
|
||||
2. Bookhoard scans all files:
|
||||
- **Calibre books**: Import from `metadata.opf`
|
||||
- **Non-Calibre books**: Use embedded metadata or filename
|
||||
|
||||
**Result**: Seamless import of both Calibre and non-Calibre books, with appropriate metadata sources for each.
|
||||
|
||||
### Example 3: Updating Calibre Metadata
|
||||
|
||||
**Scenario**: You edit metadata in Calibre (fix author name, add series, change cover).
|
||||
|
||||
**Steps**:
|
||||
1. Edit metadata in Calibre (it updates `metadata.opf`)
|
||||
2. In Bookhoard, trigger a rescan:
|
||||
- Navigate to **Admin** → **Libraries**
|
||||
- Click **Rescan** on your library
|
||||
- Or use the **Scanner API** to force rescan
|
||||
3. Bookhoard detects updated `metadata.opf` and refreshes metadata
|
||||
|
||||
**Result**: Bookhoard reflects your Calibre changes automatically.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Metadata Not Importing
|
||||
|
||||
**Problem**: Bookhoard doesn't import Calibre metadata.
|
||||
|
||||
**Solutions**:
|
||||
1. **Check file structure**: Ensure `metadata.opf` is in the same folder as the book file
|
||||
2. **Verify library type**: Ensure library type matches content (ebook vs. audiobook)
|
||||
3. **Force rescan**: Use the "Force Rescan" option to re-import all metadata
|
||||
4. **Check logs**: Review Bookhoard logs for parsing errors
|
||||
|
||||
### Incorrect Metadata
|
||||
|
||||
**Problem**: Imported metadata is wrong or incomplete.
|
||||
|
||||
**Solutions**:
|
||||
1. **Fix in Calibre**: Edit metadata in Calibre, then rescan in Bookhoard
|
||||
2. **Verify sidecar content**: Open `metadata.opf` in a text editor to check content
|
||||
3. **Check date formats**: Ensure dates are ISO format (YYYY-MM-DD)
|
||||
4. **Re-scan**: Force rescan to re-import from updated sidecar
|
||||
|
||||
### Missing Series Information
|
||||
|
||||
**Problem**: Books in a series don't show series info.
|
||||
|
||||
**Solutions**:
|
||||
1. **Check Calibre**: Verify series is set in Calibre
|
||||
2. **Check sidecar**: Ensure `calibre:series` and `calibre:series_index` are in `metadata.opf`
|
||||
3. **Re-scan**: Force rescan to update series info
|
||||
|
||||
### Performance Issues
|
||||
|
||||
**Problem**: Scanning takes a long time with Calibre library.
|
||||
|
||||
**Solutions**:
|
||||
1. **Large libraries**: Initial scan may take time; subsequent scans are faster
|
||||
2. **Network storage**: Scanning over network is slower; use local storage if possible
|
||||
3. **Watch mode**: Enable watch mode for real-time updates instead of full rescans
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Keep Calibre as Primary Source
|
||||
|
||||
**Do**:
|
||||
- ✅ Edit metadata in Calibre
|
||||
- ✅ Rescan in Bookhoard to sync changes
|
||||
- ✅ Use Calibre for library management
|
||||
|
||||
**Don't**:
|
||||
- ❌ Edit `metadata.opf` files manually
|
||||
- ❌ Modify metadata in Bookhoard if you plan to resync from Calibre
|
||||
- ❌ Keep conflicting metadata sources
|
||||
|
||||
### 2. Organize with Tags and Collections
|
||||
|
||||
- Use **tags in Calibre** for genres, moods, status (To Read, Read, etc.)
|
||||
- Create **collections in Bookhoard** for automatic organization
|
||||
- Leverage both systems for different purposes
|
||||
|
||||
### 3. Regular Syncing
|
||||
|
||||
- Set up **watch mode** in Bookhoard for automatic updates
|
||||
- Run **full rescan** after bulk edits in Calibre
|
||||
- Keep metadata consistent between Calibre and Bookhoard
|
||||
|
||||
### 4. Backup Your Library
|
||||
|
||||
- Back up your Calibre library regularly
|
||||
- Include both book files and `metadata.opf` files
|
||||
- Test restore process periodically
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Custom Calibre Columns (Future)
|
||||
|
||||
Bookhoard may support Calibre custom columns in a future release. This would allow you to import:
|
||||
- Read status
|
||||
- Favorite flags
|
||||
- Custom metadata fields
|
||||
- User-defined categories
|
||||
|
||||
Stay tuned for updates!
|
||||
|
||||
### OPDS Integration
|
||||
|
||||
You can access your Bookhoard library (including Calibre-imported books) via OPDS from Calibre-aware devices:
|
||||
- Kobo e-readers
|
||||
- KOReader
|
||||
- Phone/tablet apps (KYBook, Chunky, etc.)
|
||||
|
||||
See the [Kobo Setup Guide](devices/kobo-setup.md) or [KOReader Setup Guide](devices/koreader-setup.md) for details.
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Will Bookhoard modify my Calibre library?**
|
||||
|
||||
A: No. Bookhoard only **reads** Calibre metadata. It never modifies your Calibre library files or `metadata.opf` files.
|
||||
|
||||
**Q: Can I use both Calibre and Bookhoard?**
|
||||
|
||||
A: Yes! They're complementary:
|
||||
- Use **Calibre** for library management, conversion, and device syncing
|
||||
- Use **Bookhoard** for web access, sync across devices, and sharing
|
||||
|
||||
**Q: What if I don't use Calibre?**
|
||||
|
||||
A: Bookhoard works perfectly without Calibre. It will use embedded metadata from your book files, folder structure, and filenames.
|
||||
|
||||
**Q: Does Bookhoard import Calibre ratings?**
|
||||
|
||||
A: No. Ratings are **per-user** in Bookhoard. Calibre ratings reflect the owner's opinion, which may not match other users' opinions.
|
||||
|
||||
**Q: Can I edit Calibre metadata in Bookhoard?**
|
||||
|
||||
A: You can edit metadata in Bookhoard, but it won't sync back to Calibre. For permanent changes, edit in Calibre and rescan in Bookhoard.
|
||||
|
||||
**Q: Does this work with Calibre Web?**
|
||||
|
||||
A: Calibre Web uses the same `metadata.opf` files, so yes - Bookhoard can scan a Calibre Web library folder.
|
||||
|
||||
**Q: What about Calibre's "author sort" and "title sort"?**
|
||||
|
||||
A: Bookhoard has its own sorting logic and doesn't import Calibre sort fields. This allows for consistent sorting across all books.
|
||||
|
||||
## Resources
|
||||
|
||||
- [Calibre Website](https://calibre-ebook.com/)
|
||||
- [Calibre User Manual](https://manual.calibre-ebook.com/)
|
||||
- [Calibre Forum](https://www.mobileread.com/forums/forumdisplay.php?f=166)
|
||||
- [Bookhoard GitHub](https://github.com/yourusername/bookhoard)
|
||||
|
||||
## Need Help?
|
||||
|
||||
- **Documentation**: See [User Guide](user-guide.md) for general Bookhoard usage
|
||||
- **Troubleshooting**: See [Operations Guide](../operations/operations.md) for common issues
|
||||
- **Developer Docs**: See [Developer Portal](../developer/development.md) for technical details
|
||||
- **Issues**: Report bugs or request features on [GitHub Issues](https://github.com/yourusername/bookhoard/issues)
|
||||
@@ -27,6 +27,17 @@ Learn how to configure your e-reader devices to sync with Bookhoard:
|
||||
- Conflict resolution
|
||||
- Best practices
|
||||
|
||||
## 📚 Calibre Integration
|
||||
|
||||
- **[Calibre Integration Guide](calibre-integration.md)** - Integrating with Calibre libraries
|
||||
- What is Calibre?
|
||||
- Automatic metadata import
|
||||
- Setting up your Calibre library
|
||||
- Supported metadata and file types
|
||||
- Workflow examples
|
||||
- Troubleshooting
|
||||
- Best practices
|
||||
|
||||
## 🎨 Frontend Guide
|
||||
|
||||
**[Frontend Guide](frontend-guide.md)** - Learn how to use the Bookhoard web interface
|
||||
|
||||
Reference in New Issue
Block a user