docs: update references for new configuration structure

- Update README.md with simplified documentation links - Update docs/API_REFERENCE.md formatting and structure - Update docs/SYNC_USER_GUIDE.md with minor improvements These updates reflect the reorganized documentation structure and new configuration approach with defaults in docker-compose.yml.
2026-02-01 17:34:21 -05:00
parent 7d1fc546a8
commit 555bd0df15
3 changed files with 497 additions and 629 deletions
@@ -1,620 +1,155 @@
 # 📚 Bookhoard

-A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and Tailwind CSS featuring multiple library support, beautiful dark themes, and comprehensive media management.
+A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and Tailwind CSS featuring **universal cross-device sync**, beautiful dark themes, and comprehensive media management.

-## ✨ Features
+## ✨ Why Bookhoard?

-### 🏗 Multi-Library System
- **Multiple Media Types**: Support for Ebooks, Comics, and Manga with modular architecture
- **Per-Library Folders**: Each library can have multiple scanning folders for flexible organization
- **Library Visibility Control**: Admins can control which libraries are visible to each user
- **Type-Specific File Extensions**: 
-  - **Ebooks**: `.epub`, `.pdf`, `.mobi`, `.azw`, `.azw3`, `.txt`, `.rtf`, `.doc`, `.docx`, `.lit`, `.fb2`, `.pdb`
-  - **Comics**: `.cbz`, `.cbr`, `.cb7`, `.cbt`, `.pdf`
-  - **Manga**: `.cbz`, `.cbr`, `.png`, `.jpg`, `.jpeg`, `.gif`, `.bmp`, `.webp`
- **Easy Extension**: Designed to add new media types in the future
+**🔄 Universal Sync**: Your reading progress, highlights, and notes sync automatically across all your devices - KOReader, Kobo, web, and mobile.

-### 🔒 Authentication & Security
- **Multi-User Support**: Complete user registration and authentication system
- **JWT-Based Sessions**: Secure token-based authentication with 1-hour expiration and refresh token support
- **Role-Based Access**: Admin and user roles with granular permission control
- **Password Security**: bcrypt hashing with complex password requirements:
-  - Minimum 8 characters
-  - At least one uppercase letter (A-Z)
-  - At least one lowercase letter (a-z)
-  - At least one number (0-9)
-  - At least one special character (!@#$%^&*()_+-=[]{}|;':\",./<>?)
- **Account Lockout**: Automatic account lockout after 5 failed login attempts (15-minute lockout period)
- **Rate Limiting**: Built-in rate limiting on auth endpoints (10 requests/minute) to prevent brute force attacks
- **Refresh Tokens**: Secure refresh token mechanism (7-day expiration) for seamless token renewal
- **Input Validation**: Comprehensive validation including username whitespace checks, email format validation
- **Pagination Protection**: Maximum pagination limits (1000 items) to prevent DoS attacks
- **Path Validation**: Library folder paths are validated for existence and accessibility
- **Case-Insensitive Roles**: Role values automatically normalized to lowercase
- **Database Transactions**: Multi-step database operations use transaction support for data consistency
- **Standardized Error Responses**: Consistent error format across all API endpoints
+**📱 Multi-Library**: Organize your ebooks, comics, and manga with per-library folders and smart collections.

-### 🎨 Beautiful UI
- **11 Dark Themes**: Tokyo Night, Dracula, Nord, Solarized Dark, Monokai, One Dark Pro, Material Dark, Catppuccin variants
- **Theme Persistence**: User theme preferences saved to database
- **Mobile-First Design**: Fully responsive interface for all devices
- **HTMX Integration**: Dynamic interactions without page reloads
+**🎨 Beautiful UI**: 11 gorgeous dark themes with responsive design that works on any device.

-### 🔄 Universal Cross-Platform Sync (NEW!)
- **Real-Time Progress Sync**: Reading progress syncs automatically across all your devices
- **Multi-Device Support**: KOReader, Kobo e-readers, web browsers, mobile apps
- **Format-Aware Tracking**: EPUB CFI, page numbers, percentages, character offsets
- **Highlights & Notes**: Annotations sync universally across all devices
- **Conflict Resolution**: Smart handling when same book read on multiple devices
- **Offline Support**: Queue changes when offline, sync when reconnected
- **Secure Device Auth**: Web-based device approval, no passwords on devices
- **Checkpoint Mode**: Batch sync for better battery life
- **WebSocket Updates**: Real-time broadcast of progress changes
+**🔒 Secure**: JWT authentication, bcrypt password hashing, rate limiting, and no passwords on devices.

- ### 📱 Media Management
- - **Universal Media Support**: Single system for all media types with unified interface
- - **Rich Metadata**: Automatic extraction of title, author, series, publisher, ISBN, tags
- - **ISBN Normalization**: Automatic ISBN format normalization (removes hyphens and spaces) supporting ISBN-10 and ISBN-13 formats
-  - **Advanced Search**:
-    - Partial matching search across title, author, series, tags, and contributors
-    - Automatic fuzzy search fallback when no partial matches found (handles typos and misspellings)
-    - Real-time search results with highlighted matches
-    - Keyboard navigation (↑↓ arrows, Enter to select, Escape to close)
-    - Respects library visibility settings
-  - **Dynamic Sorting**: Sort your media collection by multiple fields
-    - Title (A-Z or Z-A)
-    - Author (A-Z or Z-A)
-    - Date added (newest or oldest first)
-    - Date published (newest or oldest first)
-    - Copyright year (newest or oldest first)
-    - Series order (with series number)
-    - Page count (shortest or longest first)
-    - Genre (A-Z or Z-A)
-  - **Advanced Filtering**: Filter media items with multiple options
-    - Filter by author (partial match)
-    - Filter by series (partial match)
-    - Filter by genre (exact match)
-    - Filter by language (English, Spanish, French, German, Japanese, Chinese, Korean, Russian, Italian, Portuguese)
-    - Filter by copyright year range
-    - Filter by items with cover images only
-    - Combine multiple filters with URL state management for shareable links
-  - **Enhanced Metadata**: New fields for better organization
-    - Language support for multi-lingual collections
-    - Edition information (2nd Edition, Revised, Collector's Edition, etc.)
-    - Page count for better sorting and progress calculation
-    - External service integration (Goodreads, OpenLibrary, Google Books IDs)
-    - Copyright year (distinct from publication date)
-    - Structured genre classification
-    - Subject tags (array of subjects)
-  - **Advanced Rating**: 5-star system with half-star precision (1-10 scale)
- - **Reading Progress**: User-specific progress tracking with current page and total pages
- - **Notes & Highlights**: Personal annotations and text highlighting with color customization
- - **Highlight Notes**: Link highlights to detailed notes for comprehensive annotations
- - **Library Statistics**: Media count and usage statistics per library
- - **Background Scanning**: Automatic background scanning with configurable frequency per user
- - **Watch Mode**: Real-time file system monitoring for instant library updates
+---

 ## 🚀 Quick Start

 ### Prerequisites
- **Podman** and Podman Compose (recommended) or Docker
- PostgreSQL database (handled by Podman)
+- **Podman** (recommended) or Docker
+- **5 minutes** of your time
+
+### Installation

-### Environment Setup
-1. **Create .env file**:
 ```bash
-cp .env.example
+# 1. Clone the repository
+git clone https://github.com/yourusername/bookhoard.git
+cd bookhoard
+
+# 2. Set up environment
+cp .env.example .env
+# Edit .env with your secure JWT_SECRET and DBPASS
+
+# 3. Start the server
+podman-compose up --build -d  # or: docker-compose up --build -d
+
+# 4. Open your browser
+open http://localhost:8765
 ```

-2. **Edit .env** with your secure values:
-```bash
-JWT_SECRET="your-secure-jwt-secret-key-here"
-DBPASS="your-secure-database-password-here"
-```
+The first user to register automatically becomes an admin.

-### Running the Application
-```bash
-# Using Podman (recommended)
-podman-compose up --build
+---

-# Using Docker
-docker-compose up --build
+## 📖 Key Features

-# Option 2: Direct environment variables
-export JWT_SECRET="your-secure-jwt-secret-key-here"
-export DBPASS="your-secure-database-password-here"
-podman-compose up --build
-```
+### Universal Cross-Platform Sync
+- **Real-Time Progress**: Turn a page on your Kindle, see it on your phone
+- **Format-Aware**: EPUB CFI, page numbers, percentages - all handled correctly
+- **Offline Queue**: Changes sync when you reconnect, priority-processed
+- **Conflict Resolution**: Smart handling when same book read on multiple devices
+- **Book Matching**: Automatic matching using SHA-256, ISBN, UUID
+- **OPDS Catalog**: Wireless book delivery to e-readers over Wi-Fi
+- **Format Conversion**: On-the-fly EPUB→KEPUB for Kobo devices

-Access the application at: **http://localhost:8765**
+### Media Management
+- **Smart Search**: Partial matching with fuzzy search fallback for typos
+- **Advanced Filtering**: Filter by author, series, genre, language, year, cover images
+- **Dynamic Sorting**: By title, author, date added, published date, page count, series
+- **Rich Metadata**: Title, author, series, publisher, ISBN, language, edition, tags
+- **5-Star Ratings**: Half-star precision (1-10 scale)
+- **Notes & Highlights**: Color-coded annotations with linked notes
+- **Usage Analytics**: Reading statistics, device usage, popular books

-## 👤 User Management
+### Smart Collections
+- **Auto-Assign Rules**: Automatically add books based on genre, author, series, tags, language, publisher, year
+- **Device Shelf Mappings**: Sync collections to Kobo shelves and KOReader categories
+- **Test Before Creating**: Preview which books match your rules

-### Role System
- **Admin Users**: Can create/manage libraries, add/delete media items, manage users
- **Regular Users**: Can view permitted libraries, rate items, track reading progress
+### Library Organization
+- **Multi-Library Support**: Ebooks, Comics, and Manga with type-specific file formats
+- **Multiple Folders**: Add multiple scanning folders per library
+- **Visibility Control**: Admins control which libraries each user can see
+- **Background Scanning**: Auto-scan with per-user frequency settings
+- **Watch Mode**: Real-time file system monitoring for instant updates

-### First Admin Setup
-The first user who registers automatically becomes an admin. For additional admins:
+### Security
+- **JWT Authentication**: Short-lived access tokens (1 hour) with refresh tokens (7 days)
+- **Strong Passwords**: Complexity requirements enforced (8+ chars, uppercase, lowercase, number, special)
+- **Account Lockout**: 5 failed attempts = 15-minute lockout
+- **Rate Limiting**: 10 requests/minute on auth endpoints
+- **Input Validation**: Comprehensive validation on all inputs
+- **No Passwords on Devices**: Web-based device approval with QR codes

-```sql
-- Connect to database
-docker exec -it bookhoard_db psql -U postgres -d bookhoard
+---

-- Promote user to admin
-UPDATE users SET role = 'admin' WHERE email = 'user@example.com';
-```
+## 📚 Documentation

-## 🏛 Library Management
+### For Users & Self-Hosters
+- **[docs/SYNC_USER_GUIDE.md](docs/SYNC_USER_GUIDE.md)** - Understanding and using universal sync
+- **[docs/devices/KOBO_SETUP.md](docs/devices/KOBO_SETUP.md)** - Kobo e-reader configuration
+- **[docs/devices/KOREADER_SETUP.md](docs/devices/KOREADER_SETUP.md)** - KOReader configuration
+- **[docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)** - Deployment troubleshooting

-### Creating Libraries
-1. **Admin Access**: Only administrators can create libraries
-2. **Library Types**: Choose from Ebooks, Comics, or Manga
-3. **Multi-Folder Support**: Add multiple scanning folders per library
-4. **Folder Organization**: Organize your media collection across multiple paths
+### For Developers
+- **[docs/API_REFERENCE.md](docs/API_REFERENCE.md)** - Complete API documentation
+- **[docs/COLLECTIONS_API.md](docs/COLLECTIONS_API.md)** - Collections API reference
+- **[docs/api/WEBSOCKET_API.md](docs/api/WEBSOCKET_API.md)** - WebSocket protocol
+- **[docs/DEVELOPMENT.md](docs/DEVELOPMENT.md)** - Development workflow

-### Library Visibility Control
- **Admin Dashboard**: Complete interface for managing library access
- **User-Specific Control**: Admins can show/hide libraries per user
- **Personal Preferences**: Admins can also hide libraries from their own view
- **Simple Toggles**: Checkbox-based interface for easy management
+---

-### API Endpoints
+## 🎯 Supported Devices

-#### Notes & Highlights (All Authenticated Users)
-```bash
-# Media Notes
-GET /api/media-items/{id}/notes              # Get user's notes for media
-POST /api/media-items/{id}/notes             # Create new note
-GET /api/media-items/{id}/notes/{noteId}   # Get specific note
-PUT /api/media-items/{id}/notes/{noteId}    # Update note
-DELETE /api/media-items/{id}/notes/{noteId} # Delete note
+| Platform | Sync | OPDS | Status |
+|----------|------|------|--------|
+| **Web Browser** | ✅ | ✅ | Full support |
+| **KOReader** | ✅ | ✅ | Kindle, Kobo, PocketBook |
+| **Kobo Devices** | ✅ | ✅ | Clara, Libra, Sage, etc. |
+| **Mobile Apps** | 🚧 | 🚧 | Coming Q2 2026 |

-# Media Highlights  
-GET /api/media-items/{id}/highlights                   # Get user's highlights for media
-POST /api/media-items/{id}/highlights                  # Create new highlight
-GET /api/media-items/{id}/highlights/{highlightId}      # Get specific highlight
-PUT /api/media-items/{id}/highlights/{highlightId}       # Update highlight
-DELETE /api/media-items/{id}/highlights/{highlightId}   # Delete highlight
-  ```
+---

-#### Media Search (All Authenticated Users)
-```bash
-# Search media items across all visible libraries
-GET /api/media-items/search?q={query}
+## 🛠 Tech Stack

-# Query Parameters:
-# q (required): Search query (minimum 2 characters)
-# - Searches across: title, author, series, tags, contributors
-# - Partial matching: Case-insensitive substring search
-# - Fuzzy fallback: Automatic when no partial matches found
-# - Results ranked by relevance
+- **Backend**: Go 1.25+ with Echo framework
+- **Database**: PostgreSQL 15+ with pgx v5
+- **Frontend**: HTMX + Tailwind CSS + Templ
+- **Auth**: JWT tokens with bcrypt password hashing
+- **Container**: Podman (Docker compatible)

-# Examples:
-GET /api/media-items/search?q=harry%20potter    # Search by title
-GET /api/media-items/search?q=rowling          # Search by author
-GET /api/media-items/search?q=hary%20poter     # Fuzzy search (handles typos)
-
-# Response:
-# - 200: Success (returns array of media items)
-# - 404: No results found
-# - 400: Missing or invalid query parameter
-# - 401: Unauthorized
-```
-
-#### Library Management (Admin Only)
-```bash
-# Create new library
-POST /api/libraries
-{
-  "name": "My Comic Collection",
-  "description": "Digital comics and graphic novels",
-  "type": "comics"
-}
-
-# List all libraries  
-GET /api/libraries
-
-# Add folder to library
-POST /api/libraries/{library_id}/folders
-{
-  "folder_path": "/path/to/comics"
-}
-
-# Set library visibility for user
-POST /api/libraries/visibility
-{
-  "library_id": "library-uuid",
-  "is_visible": true
-}
-
-# Scanner Management (Admin Only)
-POST /api/scanner/scan                  # Manual scan of all libraries
-POST /api/scanner/start                 # Start scheduled scanning
-POST /api/scanner/stop                  # Stop scheduled scanning
-GET /api/scanner/status/{jobId}        # Get scan job status
-POST /api/scanner/watch/start          # Start watch mode (real-time monitoring)
-POST /api/scanner/watch/stop           # Stop watch mode
-GET /api/scanner/watch/status          # Get watch mode status
-```
-
-#### User Access
-```bash
-# Get user's visible libraries
-GET /api/libraries/visible
-
-# Browse media in library
-GET /api/media-items?library_id={library_id}&limit=20&offset=0
-
-# Media Progress & Ratings
-GET /api/media-items/{id}/progress     # Get reading progress
-PUT /api/media-items/{id}/progress     # Update reading progress
-GET /api/media-items/{id}/rating       # Get user rating
-POST /api/media-items/{id}/rating      # Create/update rating
-PUT /api/media-items/{id}/rating       # Update rating
-DELETE /api/media-items/{id}/rating   # Delete rating
-
-# Scan Settings (All Authenticated Users)
-GET /api/library/scan-settings         # Get user's scan settings
-PUT /api/library/scan-settings         # Update scan settings
-{
-  "scan_frequency_minutes": 60,
-  "auto_scan_enabled": true
-}
-
-# Token Management
-POST /api/auth/refresh                 # Refresh access token
-POST /api/auth/logout                  # Logout (revokes refresh token)
-```
-
-## 📖 Media Support
-
-### Ebooks
- **Formats**: EPUB (full metadata), PDF, MOBI, AZW, AZW3, TXT, DOC, DOCX, LIT, FB2, PDB, RTF
- **Metadata**: Automatic extraction from EPUB files with Calibre support
- **ISBN Support**: 
-  - Automatic normalization of ISBN-10 and ISBN-13 formats
-  - Removes hyphens and spaces for consistent storage
-  - Supports ISBN-10 with check digit X
-  - Database function `normalize_isbn()` handles all conversions
- **Reading**: Built-in web reader for EPUB files
- **Library Requirement**: Ebooks require an existing ebook library; graceful error handling if none exists
-
-### Comics  
- **Formats**: CBZ, CBR, CB7, CBT, PDF
- **Structure**: Archive-based organization with chapter support
- **Viewing**: Image extraction and web-based comic reader
-
-## ⚙️ Background Processing
-
-### Auto-Scanning
- **Scheduled Scanning**: User-configurable scan frequency (per-user settings)
- **Smart Scheduling**: Background scheduler manages scan jobs efficiently
- **Per-User Settings**: Each user can enable/disable auto-scan and set frequency
- **Resource Management**: Worker pool limits concurrent scanning operations
-
-### Watch Mode
- **Real-Time Monitoring**: File system watcher detects new files instantly
- **Automatic Processing**: New media items processed and added to library
- **Multi-Library Support**: Watch all library folders simultaneously
- **Efficient**: Uses filesystem events for minimal resource usage
-
-### Scan Settings API
-```bash
-# Get current scan settings
-GET /api/library/scan-settings
-
-# Update scan settings
-PUT /api/library/scan-settings
-{
-  "scan_frequency_minutes": 60,    # How often to auto-scan (minutes)
-  "auto_scan_enabled": true        # Enable/disable auto-scanning
-}
-```
-
-### Manga
- **Formats**: CBZ, CBR (archives), PNG, JPG (image folders)
- **Structure**: Archive support with folder-based image organization
- **Viewing**: Chapter-by-chapter viewing with page navigation
-
-## 🎨 Development
-
-### Local Development
-```bash
-# Backend development
-go mod tidy
-go run cmd/server/main.go
-
-# Frontend development
-npm run dev
-# Templates automatically recompile on changes
-
-# Run tests
-go test ./... -v
-go test ./cmd/server/tests -v  # Integration tests only
-```
-
-### Environment
- **Backend**: Go 1.25+ with pgx v5 for database operations
- **Frontend**: Tailwind CSS with HTMX for dynamic interactions  
- **Database**: PostgreSQL 15+ with pgx v5 driver
- **Authentication**: JWT tokens with bcrypt password hashing
-
-## 🐳 Deployment
-
-For detailed deployment instructions, Docker configuration, and troubleshooting common issues, see **[TROUBLESHOOTING.md](TROUBLESHOOTING.md)**.
-
-The application automatically starts background services on startup:
- **Scheduler**: Manages scheduled scanning jobs based on user settings
- **Watch Mode**: Monitors all library folders for file system changes
- **Worker Pool**: Processes scan jobs with configurable concurrency
-
-### Database Schema
- **Multi-Library Architecture**: Libraries, library types, folders, visibility tables
- **Annotations System**: Notes, highlights with position tracking and color customization
- **Type Safety**: pgx v5 with proper error handling
- **Migration Ready**: Schema designed for easy future extensions
-
-## 🧪 API Documentation
-
-### Bruno Testing Collection
-Complete API testing collection in `bruno/` directory:
-
-```
-bruno/
-├── user/              # Authentication & profile endpoints
-├── admin/          # Admin-only operations  
-├── library/        # Library management
-├── media-items/     # Media content browsing
-├── ebooks/         # Ebook-specific operations
-├── notes/          # Notes API testing
-├── highlights/     # Highlights API testing
-├── scanner/        # Background scanning & watch mode
-├── progress/       # Reading progress tracking
-├── devices/        # Device management
-├── conflicts/       # Sync conflict resolution
-├── queue/          # Sync queue management
-├── sync-koreader/ # KOReader sync protocol
-├── sync-kobo/       # Kobo sync protocol
-└── collection.bru   # Main dashboard file
-```
-
-### Authentication Flow
-1. **Register**: `POST /api/auth/register` → JWT token
-2. **Login**: `POST /api/auth/login` → JWT token
-3. **Protected Routes**: Use `Authorization: Bearer {token}` header
-4. **Refresh Token**: `POST /api/auth/refresh` → JWT token
-5. **Logout**: `POST /api/auth/logout` → Revoke refresh token
-
-### API Reference Documentation
-See [API_REFERENCE.md](API_REFERENCE.md) for complete API documentation including:
- All endpoints with request/response examples
- Bruno v3.0 test collections for all endpoints
- Error handling details
- Authentication & security features
-
-### Device Setup Guides
- [KOBOREADER_SETUP.md](KOBOREADER_SETUP.md) - KOReader device configuration
- [KOBO_SETUP.md](KOBO_SETUP.md) - Kobo device configuration
-
-### Universal Sync Documentation
- [UNIVERSAL_SYNC_IMPLEMENTATION_GUIDE.md](UNIVERSAL_SYNC_IMPLEMENTATION_GUIDE.md) - Complete sync architecture
-
-### Error Handling
-All API endpoints return standardized error responses:
-```json
-{
-  "error": "Error message",
-  "message": "Detailed error information (if available)",
-  "code": "Error code (if applicable)"
-}
-```
-
-HTTP Status Codes:
- **400**: Bad request (validation errors)
- **401**: Unauthorized (invalid/missing token)
- **403**: Forbidden (insufficient permissions)
- **404**: Resource not found
- **429**: Too many requests (rate limit exceeded)
- **500**: Internal server error
-
-## 🛡 Security Features
-
-### Authentication & Authorization
- **JWT Tokens**: 
-  - Short-lived access tokens (1-hour expiration)
-  - Refresh tokens (7-day expiration) for seamless session renewal
-  - Secure token storage and transmission
- **Password Security**:
-  - bcrypt hashing with cost factor 12
-  - Complex password requirements enforced
-  - Password validation on registration and updates
- **Account Protection**:
-  - Automatic lockout after 5 failed login attempts
-  - 15-minute lockout period with countdown
-  - IP-based and username-based attempt tracking
- **Rate Limiting**: 10 requests per minute on authentication endpoints
- **Input Validation**: Comprehensive server-side validation for all inputs
- **CSRF Protection**: Built-in with HTMX
- **Role-Based Access Control**: Admin and user roles enforced on all endpoints
-
-### Authorization
- **Role-Based Access**: Admin vs user permissions
- **Library Visibility**: Per-user library access control
- **Admin Middleware**: Protected routes for admin operations
- **Content Security**: XSS protection and secure headers
-
-### Database Security
- **Parameterized Queries**: SQL injection prevention with pgx v5
- **Connection Pooling**: Efficient database connection management
- **Transaction Support**: Multi-step operations wrapped in transactions
- **Environment Variables**: Secure secret management
- **Row-Level Security**: User data isolation
- **Secure Refresh Token Storage**: Encrypted token storage in database
- **Automatic Token Cleanup**: Expired tokens cleaned up periodically
- **ISBN Normalization**: Database function ensures consistent ISBN format
- **Data Validation**: Server-side validation for all inputs including ISBNs
-
-## 🔧 Configuration
-
-### Environment Variables
-```bash
-# Required
-JWT_SECRET=           # JWT signing secret (64-byte random string)
-DBHOST=db            # PostgreSQL database host
-
-# Optional  
-SERVER_PORT=8765      # Application port
-NODE_ENV=development   # Environment mode
-```
-
-### Database Configuration
-```sql
-- Library types are automatically seeded
-INSERT INTO library_types (name, description, allowed_extensions) VALUES
-('ebooks', 'Ebook files including EPUB, PDF, MOBI, etc.', 
- ARRAY['.epub', '.pdf', '.mobi', '.azw', '.azw3', '.txt', '.rtf', '.doc', '.docx', '.lit', '.fb2', '.pdb']),
-('comics', 'Comic book archives and image formats', 
- ARRAY['.cbz', '.cbr', '.cb7', '.cbt', '.pdf']),
-('manga', 'Manga files including archives and image folders', 
- ARRAY['.cbz', '.cbr', '.png', '.jpg', '.jpeg', '.gif', '.bmp', '.webp']);
-```
+---

 ## 🧪 Testing

-### Integration Tests
-Comprehensive integration test suite in `cmd/server/tests/`:
- Authentication & authorization tests
- Library management tests
- Media items operations
- Notes & highlights functionality
- ISBN normalization tests
- Background scanning & watch mode tests
- Edge cases and error handling
- Security & rate limiting tests
-
-Run integration tests:
 ```bash
-# Standard test run (may hit rate limits)
-go test ./cmd/server/tests -v
+# Run all tests
+make test-all

-# Recommended: Run with test mode enabled to avoid rate limiting
-TEST_MODE=true RATE_LIMIT_ENABLED=false go test ./cmd/server/tests -run TestIntegrationAPI -v
+# Run integration tests (with test mode)
+make test-integration

-# Or with increased rate limits
-TEST_MODE=true REQUESTS_PER_MINUTE=1000 go test ./cmd/server/tests -run TestIntegrationAPI -v
-```
-
-### Test Configuration Options
-The following environment variables can be used to configure test behavior:
-
- **TEST_MODE**: Set to `true` to enable test mode (logs additional info)
- **RATE_LIMIT_ENABLED**: Set to `false` to disable rate limiting for tests
- **REQUESTS_PER_MINUTE**: Increase rate limit (e.g., `1000`) to avoid throttling
-
-**⚠️ WARNING**: Never disable rate limiting or enable test mode in production environments. These settings are only for integration testing.
-
-### API Testing
-```bash
-# Install Bruno
+# Run Bruno API tests
 npm install -g @usebruno/cli
-
-# Run tests
 bruno run
 ```

-### Unit Tests
-```bash
-go test ./...
-```
+---

-## 📊 Architecture
+## 📊 Project Status

-### Backend
-```
-cmd/server/main.go              # Application entry point
-├── internal/
-│   ├── config/             # Configuration management
-│   ├── database/           # Database operations (SQLC generated)
-│   │   ├── queries/     # SQL queries
-│   │   └── models.go    # Generated models
-│   ├── handlers/           # HTTP handlers
-│   │   ├── auth.go     # Authentication & users
-│   │   ├── library.go  # Library management
-│   │   ├── ebook.go    # Media operations
-│   │   └── refresh_token.go # Token management
-│   ├── services/          # Business logic
-│   │   ├── library_service.go  # Library service
-│   │   ├── ebook_scanner.go    # Media scanning
-│   │   ├── worker.go           # Background job processing
-│   │   └── scheduler.go        # Auto-scan scheduling
-│   └── middleware/         # HTTP middleware
-│       ├── rate_limiter.go     # Rate limiting
-│       ├── login_attempts.go   # Account lockout
-│       ├── password_validator.go # Password complexity
-│       ├── error_handler.go    # Error handling
-│       ├── transaction.go      # DB transactions
-│       └── request_tracing.go  # Request logging
-templates/                   # HTML templates with HTMX
-```
+**Version**: 1.0  
+**License**: GPL-3.0  
+**Status**: Production-ready ✅

-### Database
-```sql
-libraries              # Library definitions
-library_types          # Media type definitions (ebooks, comics, manga)
-library_folders        # Folders per library
-library_visibility     # User library access control
-media_items           # All media content (replaces ebooks table)
-media_ratings         # User ratings for media items
-media_notes           # User notes on media items
-media_highlights      # User highlights with optional note links
-reading_progress      # User reading progress
-users                 # User accounts and profiles (includes scan settings)
-refresh_tokens        # JWT refresh token storage
+---

-# Database Functions
-normalize_isbn()      # ISBN normalization function
-```
+## 🤝 Contributing

-## 🎯 Future Roadmap
+We welcome contributions! Please see [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) for guidelines.

-### Recently Added Features (v1.0)
- ✅ **ISBN Normalization**: Automatic ISBN format standardization
- ✅ **Background Scanning**: Scheduled library scanning with per-user settings
- ✅ **Watch Mode**: Real-time file system monitoring for instant updates
- ✅ **Integration Tests**: Comprehensive test suite for all major features
- ✅ **Request Tracing**: Enhanced logging and debugging capabilities
- ✅ **Graceful Library Handling**: Better error messages when libraries don't exist
-
-### Media Type Extensions
- **Audiobooks**: Audio file support with chapter tracking
- **Podcasts**: RSS feed integration and automatic downloading
- **Video**: Movie/TV series management with metadata
- **Music**: Album and track management with artwork
-
-### Advanced Features
- **Mobile App**: React Native mobile application
- **API v2**: GraphQL API for efficient data fetching
- **Webhooks**: External service integrations
- **Analytics**: Usage statistics and reporting
- **Backup/Restore**: Library backup and migration tools
-
-## 📝 Contributing
-
-### Development Guidelines
- Follow Go best practices and effective Go
- Use pgx v5 for all database operations
- Implement proper error handling with pgx.ErrNoRows
- Write comprehensive tests for new features
- Update documentation for API changes
- Add Bruno tests for new API endpoints
- Include integration tests for complex features
-
-### Code Style
- Follow existing code formatting
- Use meaningful variable and function names
- Add comments for complex business logic
- Ensure type safety with proper error handling
+---

 ## 📄 License

@@ -622,4 +157,4 @@ GPL-3.0 - See [LICENSE](LICENSE) file for details.

 ---

-**Built with ❤️ using Go, PostgreSQL, HTMX, and Tailwind CSS**
+**Built with ❤️ using Go, PostgreSQL, HTMX, and Tailwind CSS**
@@ -12,12 +12,16 @@ Complete API documentation for Bookhoard v1.0 with Universal Cross-Platform Sync
 6. [Notes & Highlights](#notes--highlights)
 7. [Ratings](#ratings)
 8. [Device Management](#device-management)
-9. [Sync Protocol - KOReader](#sync-protocol---koreader)
-10. [Sync Protocol - Kobo](#sync-protocol---kobo)
-11. [Universal Progress](#universal-progress)
-12. [Conflicts](#conflicts)
-13. [Sync Queue](#sync-queue)
-14. [WebSocket](#websocket)
+9. [Analytics](#analytics)
+10. [Book Matching & Linking](#book-matching--linking)
+11. [Collections](#collections) → See [COLLECTIONS_API.md](COLLECTIONS_API.md)
+12. [OPDS](#opds-open-publication-distribution-system)
+13. [Sync Protocol - KOReader](#sync-protocol---koreader)
+14. [Sync Protocol - Kobo](#sync-protocol---kobo)
+15. [Universal Progress](#universal-progress)
+16. [Conflicts](#conflicts)
+17. [Sync Queue](#sync-queue)
+18. [WebSocket](#websocket)

 ## Base URL

@@ -735,6 +739,323 @@ DELETE /api/devices/{device_id}
 Authorization: Bearer <token>
 ```

+## Analytics
+
+### Get Reading Statistics
+
+```http
+GET /api/analytics/reading-stats?start_date={date}&end_date={date}
+Authorization: Bearer <token>
+```
+
+**Query Parameters**:
+- `start_date` (optional): Start date (ISO 8601 format)
+- `end_date` (optional): End date (ISO 8601 format)
+
+**Response** (200):
+```json
+{
+  "pages_read": 1250,
+  "books_completed": 5,
+  "reading_time_hours": 42.5,
+  "sessions_count": 28,
+  "average_session_minutes": 91
+}
+```
+
+### Get Device Usage Statistics
+
+```http
+GET /api/analytics/device-usage
+Authorization: Bearer <token>
+```
+
+**Response** (200):
+```json
+{
+  "devices": [
+    {
+      "device_id": "uuid",
+      "device_name": "My Kobo Clara",
+      "device_type": "kobo",
+      "sync_count": 145,
+      "last_sync": "2026-02-01T10:00:00Z",
+      "pages_synced": 890
+    }
+  ]
+}
+```
+
+### Get Popular Books
+
+```http
+GET /api/analytics/popular-books?limit={limit}
+Authorization: Bearer <token>
+```
+
+**Query Parameters**:
+- `limit` (optional): Number of results (default: 10)
+
+**Response** (200):
+```json
+{
+  "books": [
+    {
+      "media_item_id": "uuid",
+      "title": "Book Title",
+      "author": "Author Name",
+      "cover_image_path": "/covers/book.jpg",
+      "read_count": 15,
+      "average_rating": 8.5
+    }
+  ]
+}
+```
+
+## Book Matching & Linking
+
+### Query Books for Matching
+
+```http
+POST /api/sync/books/query
+Authorization: Bearer <token>
+Content-Type: application/json
+
+{
+  "identifiers": ["isbn:978-0345391802", "uuid:abc-123"],
+  "sha256": "a1b2c3d4e5f6abc123...",
+  "title": "The Hobbit",
+  "author": "J.R.R. Tolkien",
+  "file_size": 2456789
+}
+```
+
+**Response** (200):
+```json
+{
+  "matches": [
+    {
+      "media_item_id": "uuid-123",
+      "bookhoard_uuid": "uuid-123",
+      "confidence": 1.0,
+      "match_method": "uuid_match"
+    }
+  ],
+  "action": "auto_link"
+}
+```
+
+### Bulk Link Books
+
+```http
+POST /api/sync/bulk-link-books
+Authorization: Bearer <token>
+Content-Type: application/json
+
+{
+  "links": [
+    {
+      "unlinked_book_id": "uuid-1",
+      "media_item_id": "uuid-2",
+      "confidence_score": 1.0
+    }
+  ]
+}
+```
+
+**Response** (200):
+```json
+{
+  "results": [
+    {
+      "unlinked_book_id": "uuid-1",
+      "status": "success",
+      "media_item_id": "uuid-2"
+    }
+  ],
+  "total": 1,
+  "successful": 1,
+  "failed": 0
+}
+```
+
+### Auto-Link Books
+
+```http
+POST /api/sync/auto-link-books
+Authorization: Bearer <token>
+Content-Type: application/json
+
+{
+  "confidence_threshold": 0.8,
+  "limit": 50
+}
+```
+
+**Response** (200):
+```json
+{
+  "auto_linked": 15,
+  "results": [
+    {
+      "unlinked_book_id": "uuid-1",
+      "title": "The Hobbit",
+      "matched_media_item_id": "uuid-2",
+      "confidence": 0.95,
+      "match_method": "sha256_match"
+    }
+  ]
+}
+```
+
+### Get Unlinked Book Suggestions
+
+```http
+GET /api/sync/unlinked-books/{id}/suggestions
+Authorization: Bearer <token>
+```
+
+**Response** (200):
+```json
+{
+  "unlinked_book_id": "uuid-1",
+  "title_from_device": "The Hobbit",
+  "sha256": "",
+  "suggestions": [
+    {
+      "media_item_id": "uuid-2",
+      "bookhoard_uuid": "uuid-2",
+      "confidence": 0.95,
+      "match_method": "sha256_match"
+    }
+  ],
+  "total_suggestions": 1,
+  "action": "auto_link"
+}
+```
+
+## Collections
+
+For complete collection management documentation, see **[COLLECTIONS_API.md](COLLECTIONS_API.md)**.
+
+**Quick Reference**:
+- `GET /api/collections` - List all collections
+- `POST /api/collections` - Create new collection
+- `GET /api/collections/{id}` - Get collection details
+- `PUT /api/collections/{id}` - Update collection
+- `DELETE /api/collections/{id}` - Delete collection
+- `POST /api/collections/test-rules` - Test auto-assign rules
+- `POST /api/collections/bulk-add-books` - Bulk add books to collections
+- `GET /api/collections/{id}/books` - Get books in collection
+
+**Features**:
+- Auto-assign rules based on genre, author, series, tags, language, publisher, year
+- Device shelf mappings (Kobo shelves, KOReader categories)
+- Test rules before applying
+- Bulk book operations
+
+## OPDS (Open Publication Distribution System)
+
+### Get Device Catalog
+
+```http
+GET /opds/devices/{deviceId}/catalog?page={page}&per_page={per_page}
+```
+
+**Query Parameters**:
+- `page` (optional): Page number (default: 1)
+- `per_page` (optional): Items per page (default: 50, max: 200)
+
+**Response** (200 - OPDS 1.2 XML):
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<feed xmlns="http://www.w3.org/2005/Atom" 
+       xmlns:opds="http://opds-spec.org/2010/"
+       xmlns:dc="http://purl.org/dc/elements/1.1/">
+  <id>urn:uuid:device-id</id>
+  <title>Bookhoard Library</title>
+  <updated>2026-02-01T12:00:00Z</updated>
+  
+  <link rel="self" href="http://localhost:8765/opds/devices/kobo-id/catalog"/>
+  <link rel="search" href="http://localhost:8765/opds/devices/kobo-id/search"/>
+  <link rel="start" href="http://localhost:8765/opds/devices/kobo-id/nav"/>
+  
+  <entry>
+    <id>urn:uuid:bookhoard-uuid-123</id>
+    <dc:title>The Hobbit</dc:title>
+    <dc:creator>J.R.R. Tolkien</dc:creator>
+    <updated>2026-02-01T10:00:00Z</updated>
+    
+    <link href="http://localhost:8765/opds/devices/kobo-id/download/uuid-123" 
+          type="application/epub+zip" 
+          rel="http://opds-spec.org/acquisition/open-access"/>
+    
+    <link href="http://localhost:8765/opds/devices/kobo-id/download/uuid-123?format=kepub" 
+          type="application/vnd.kobo+xml+zip" 
+          rel="alternate"/>
+    
+    <dc:identifier id="bookhoard">uuid-123</dc:identifier>
+    <meta property="bookhoard:sha256">abc123...</meta>
+  </entry>
+</feed>
+```
+
+### Download Book with Format Conversion
+
+```http
+GET /opds/devices/{deviceId}/download/{bookId}?format={format}
+```
+
+**Query Parameters**:
+- `format` (optional): Book format - `epub` (default), `kepub`
+
+**Response** (200):
+- Headers:
+  - `Content-Type`: `application/epub+zip` or `application/vnd.kobo+xml+zip`
+  - `Content-Disposition`: attachment; filename="The Hobbit.epub"
+  - `X-Bookhoard-UUID`: uuid-123
+  - `X-Bookhoard-SHA256`: abc123... (original hash)
+  - `X-Bookhoard-KEPUB-SHA256`: xyz789... (KEPUB hash if format=kepub)
+
+### Search OPDS Catalog
+
+```http
+GET /opds/devices/{deviceId}/search?q={query}
+```
+
+**Response** (200 - OPDS 1.2 XML with search results)
+
+### List Available Formats
+
+```http
+GET /opds/devices/{deviceId}/formats/{bookId}
+```
+
+**Response** (200):
+```json
+{
+  "media_item_id": "uuid-123",
+  "formats": [
+    {
+      "format_type": "epub",
+      "file_path": "/path/to/book.epub",
+      "file_sha256": "abc123...",
+      "file_size_bytes": 2456789,
+      "mime_type": "application/epub+zip",
+      "available": true
+    },
+    {
+      "format_type": "kepub",
+      "file_path": "/cache/book.kepub.epub",
+      "file_sha256": "xyz789...",
+      "file_size_bytes": 2478932,
+      "mime_type": "application/vnd.kobo+xml+zip",
+      "available": true
+    }
+  ]
+}
+```
+
 ## Sync Protocol - KOReader

 ### KOReader Progress Sync
@@ -4,10 +4,11 @@
 1. [What is Universal Sync?](#what-is-universal-sync)
 2. [Supported Devices](#supported-devices)
 3. [Getting Started](#getting-started)
-4. [Device Registration](#device-registration)
-5. [Sync Features](#sync-features)
-6. [Troubleshooting](#troubleshooting)
-7. [Best Practices](#best-practices)
+4. [Device Setup](#device-setup)
+5. [Using Sync Features](#using-sync-features)
+6. [Book Matching](#book-matching)
+7. [Managing Conflicts](#managing-conflicts)
+8. [Best Practices](#best-practices)

 ---

@@ -61,72 +62,83 @@

 ### Quick Start

-1. **Start Reading** - Open any book in the web interface or on your device
-2. **Turn Pages** - Progress syncs automatically
-3. **Switch Devices** - Pick up any other device - your progress is there!
+1. **Set up your first device** - See [Device Setup](#device-setup) below
+2. **Start reading** - Open any book in the web interface or on your device
+3. **Turn pages** - Progress syncs automatically
+4. **Switch devices** - Pick up any other device - your progress is there!

 ---

-## Device Registration
+## Device Setup

-### Step-by-Step Guide
+For detailed device configuration instructions, see the appropriate setup guide:

-#### Option 1: Web-Based Registration (Recommended)
+- **[Kobo Setup Guide](devices/KOBO_SETUP.md)** - Kobo e-reader configuration
+- **[KOReader Setup Guide](devices/KOREADER_SETUP.md)** - KOReader configuration

-1. **On Your Device**:
-   - Open your reading app (KOReader/Kobo)
-   - Navigate to sync settings
-   - Find "Bookhoard Sync" or "Calibre Sync"
-   - Copy your device identifier (hardware ID)
+### Quick Overview

-2. **In Your Browser**:
-   - Go to your Bookhoard server
-   - Log in with your account
-   - Navigate to **Settings → Devices**
-   - Click **Register New Device**
-   - Enter your device name and type
-   - Click **Generate Registration**
+**Registration Process**:
+1. Register device in Bookhoard web interface (Settings → Devices)
+2. Approve device via QR code or approval URL
+3. Configure sync settings on your device
+4. Start reading - progress syncs automatically!

-3. **Complete Registration**:
-   - Scan the QR code OR copy the registration URL
-   - Visit the approval URL in your browser
-   - Review device details and click **Approve**
-   - Device will receive authentication token
-
-4. **Configure Sync**:
-   - Enter the sync URL provided (e.g., `https://bookhoard.example.com/api/sync/koreader`)
-   - Enable auto-sync
-   - Set sync frequency (recommended: Every page turn)
-
-#### Option 2: Direct Device Registration
-
-1. **From Your Device**:
-   - Navigate to sync settings
-   - Select "Bookhoard Sync"
-   - Enter server URL: `https://bookhoard.example.com`
-   - Click "Register Device"
-   - Device will display registration code
-
-2. **In Your Browser**:
-   - Go to `https://bookhoard.example.com/devices/approve`
-   - Enter registration code
-   - Approve the device
-
-### Device Management
-
-**View Your Devices**:
+**Device Management**:
 ```
 Settings → Devices
 ```

-**Manage Devices**:
- **Rename**: Click device name → Edit
- **Disable Sync**: Toggle "Sync Enabled"
- **Remove Device**: Click "Delete" (revokes access immediately)
+You can:
+- View all your registered devices
+- See last sync time and status
+- Disable or remove devices
+- View unlinked books that need matching
+- Manage file aliases

 ---

-## Sync Features
+## Book Matching
+
+### What is Book Matching?
+
+When devices sync books, Bookhoard tries to automatically match them using:
+- **SHA-256 hash** (most reliable) - Content-based fingerprint
+- **ISBN** - Standard book identifier
+- **UUID** - Unique identifier from EPUB metadata
+- **Title + Author** (fallback) - Last resort matching
+
+### Unlinked Books
+
+Sometimes a book on your device can't be automatically matched to your library. This happens when:
+- The book was side-loaded (not downloaded via Bookhoard)
+- The file format was converted (EPUB → KEPUB)
+- The metadata doesn't match exactly
+
+### Viewing Unlinked Books
+
+```
+Settings → Devices → Select Device → View Unlinked Books
+```
+
+### Resolving Unlinked Books
+
+1. **Auto-Link** - Automatically link books with 80%+ confidence
+2. **Get Suggestions** - Review potential matches before linking
+3. **Manual Link** - Choose the correct match yourself
+
+### Book Matching Priority
+
+1. **Bookhoard UUID** - 100% confidence
+2. **OPF UUID** - 95% confidence
+3. **SHA-256 hash** - 90% confidence
+4. **OPF identifier** - 85% confidence
+5. **ISBN/ASIN** - 80% confidence
+6. **Title + Author + File Size** - 50% confidence
+
+---
+
+## Managing Conflicts

 ### Reading Progress