John O'Keefe
72839921f2
.IMAGE — extensions removed from AllowedExtensions (scan gate), bookExtensions, MimeTypes and sync/format.go maps: .doc/.lit files are no longer scanned or indexed. classifyFormatGroup gains the RTF and PDB reflowable arms that were missing when those rungs shipped (rows only reclassify on creation). Scanner docs updated. DOC + LIT have no credible JS tooling (mammoth is docx-only; LIT needs LZX and its DRM variants are dead) and text-extraction-only support would misrepresent what the reader can do. Verified: probe .doc/.lit files watched but never scanned; probe .txt control scanned; TestClassifyFormatGroup extended and passing. Docs change granted in-session by the user (extensions + format docs).
173 lines
4.8 KiB
Markdown
173 lines
4.8 KiB
Markdown
# Library Scanner Overview
|
|
|
|
The Bookhoard scanner provides comprehensive library management for ebooks, comics, and manga with both manual and automated scanning capabilities.
|
|
|
|
## Scanner Types
|
|
|
|
### Manual Scanning
|
|
|
|
- **One-time scan**: On-demand scanning of library folders
|
|
- **Progress tracking**: Real-time status updates with file-by-file progress
|
|
- **Error reporting**: Detailed logs of failed files with error messages
|
|
|
|
### Automated Scanner
|
|
|
|
- **Periodic scanning**: Configurable interval-based background scanning
|
|
- **Automatic discovery**: Detects new files without manual intervention
|
|
- **Resource-efficient**: Minimizes system impact with smart scheduling
|
|
|
|
### Watch Mode
|
|
|
|
- **Real-time monitoring**: Instant detection of file system changes
|
|
- **Event-driven**: Processes files immediately upon addition/modification
|
|
- **Platform support**: Works with inotify (Linux), FSEvents (macOS), and ReadDirectoryChangesW (Windows)
|
|
|
|
## Supported Formats
|
|
|
|
### Ebooks
|
|
|
|
| Format | Extensions |
|
|
| -------- | ------------------------ |
|
|
| EPUB | `.epub` |
|
|
| PDF | `.pdf` |
|
|
| Kindle | `.mobi` |
|
|
| Text | `.txt`, `.rtf` |
|
|
| Document | `.docx` |
|
|
| Other | `.fb2`, `.pdb` |
|
|
|
|
DOC (Word 97-2003 binary) and LIT (Microsoft Reader) are deliberately
|
|
unsupported: no credible JavaScript parser exists for either, and
|
|
compromising on fidelity is against the project's format goals. Files
|
|
with these extensions are not scanned.
|
|
|
|
### Comics
|
|
|
|
| Format | Extensions | Archive Type |
|
|
| -------------- | ---------- | ------------ |
|
|
| Comic Book ZIP | `.cbz` | ZIP |
|
|
| Comic Book RAR | `.cbr` | RAR |
|
|
| Comic Book 7z | `.cb7` | 7-Zip |
|
|
| Comic Book TAR | `.cbt` | TAR |
|
|
| PDF Comics | `.pdf` | PDF |
|
|
|
|
### Manga
|
|
|
|
| Format | Extensions | Notes |
|
|
| -------------- | ------------------------------------------------ | ---------------- |
|
|
| Comic Archives | `.cbz`, `.cbr` | Same as comics |
|
|
| Image Files | `.png`, `.jpg`, `.jpeg`, `.gif`, `.bmp`, `.webp` | Individual pages |
|
|
|
|
## Metadata Extraction
|
|
|
|
### Comic Archives (.cbz, .cbr, .cb7, .cbt)
|
|
|
|
The scanner automatically extracts metadata from comic archives:
|
|
|
|
**ComicInfo.xml Support:**
|
|
|
|
- Series title
|
|
- Issue number
|
|
- Publisher
|
|
- Writer, artist, inker, colorist
|
|
- Year, month
|
|
- Genre, tags
|
|
- Cover image extraction
|
|
|
|
**Fallback Metadata:**
|
|
|
|
- Filename parsing
|
|
- Archive structure analysis
|
|
- Page count detection
|
|
|
|
### Manga Processing
|
|
|
|
- **Archive-based**: Processes .cbz/.cbr files like comics
|
|
- **Image-based**: Handles directories of sequential images
|
|
- **Chapter detection**: Identifies chapter/volume numbers from filenames
|
|
- **Series grouping**: Groups images into logical manga volumes
|
|
|
|
## Scanner Features
|
|
|
|
### Smart Deduplication
|
|
|
|
- SHA256 hash calculation for all files
|
|
- Automatic duplicate detection and skipping
|
|
- Efficient incremental updates
|
|
|
|
### Library Type Awareness
|
|
|
|
- Format filtering based on library type
|
|
- Type-specific metadata extraction
|
|
- Appropriate thumbnail generation
|
|
|
|
### Error Handling
|
|
|
|
- Continues on individual file errors
|
|
- Detailed error reporting in scan status
|
|
- Failed file tracking for retry
|
|
|
|
### Progress Tracking
|
|
|
|
- Total files vs. processed files
|
|
- Percentage completion
|
|
- Added, updated, and failed file counts
|
|
- Per-file error messages
|
|
|
|
## Performance Considerations
|
|
|
|
### Large Libraries
|
|
|
|
- **Scanning speed**: Processes hundreds of files per second
|
|
- **Memory usage**: Streaming metadata extraction
|
|
- **Database efficiency**: Batch inserts and updates
|
|
|
|
### Resource Limits
|
|
|
|
- **Configurable intervals**: Prevent excessive scanning
|
|
- **Rate limiting**: Watch mode debounce settings
|
|
- **Admin controls**: Start/stop operations as needed
|
|
|
|
## Usage Examples
|
|
|
|
### Create and Scan a Comic Library
|
|
|
|
```json
|
|
POST /api/libraries
|
|
{
|
|
"name": "My Comic Collection",
|
|
"library_type_id": "comics-type-uuid",
|
|
"description": "Marvel and DC comics"
|
|
}
|
|
|
|
POST /api/scanner/scan
|
|
{
|
|
"library_id": "library-uuid",
|
|
"recursive": true
|
|
}
|
|
```
|
|
|
|
### Enable Watch Mode for Manga
|
|
|
|
```json
|
|
POST /api/scanner/watch/start
|
|
{
|
|
"library_id": "manga-library-uuid"
|
|
}
|
|
```
|
|
|
|
### Check Scan Progress
|
|
|
|
```http
|
|
GET /api/scanner/status/550e8400-e29b-41d4-a716-446655440000
|
|
```
|
|
|
|
## Related Endpoints
|
|
|
|
- [Scan Library](scan_library.md) - Manual one-time scan
|
|
- [Start Scanner](start_scanner.md) - Automated periodic scanning
|
|
- [Stop Scanner](stop_scanner.md) - Stop automated scanner
|
|
- [Get Scan Status](get_scan_status.md) - Check scan progress
|
|
- [Start Watch Mode](start_watch_mode.md) - Enable real-time monitoring
|
|
- [Stop Watch Mode](stop_watch_mode.md) - Disable watch mode
|
|
- [Get Watch Mode Status](get_watch_mode_status.md) - Check watch status
|