Bookhoard/bookhoard
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 12 Wiki Activity

docs: update references for new configuration structure

Browse Source 
- Update README.md with simplified documentation links
- Update docs/API_REFERENCE.md formatting and structure
- Update docs/SYNC_USER_GUIDE.md with minor improvements

These updates reflect the reorganized documentation structure
and new configuration approach with defaults in docker-compose.yml.
This commit is contained in:
john-okeefe
2026-02-01 17:34:21 -05:00
parent 7d1fc546a8
commit 555bd0df15
3 changed files with 497 additions and 629 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+103 -568
View File
@@ -1,620 +1,155 @@
# 📚 Bookhoard # 📚 Bookhoard
A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and Tailwind CSS featuring multiple library support, beautiful dark themes, and comprehensive media management. A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and Tailwind CSS featuring **universal cross-device sync**, beautiful dark themes, and comprehensive media management.
## ✨ Features ## ✨ Why Bookhoard?
### 🏗 Multi-Library System **🔄 Universal Sync**: Your reading progress, highlights, and notes sync automatically across all your devices - KOReader, Kobo, web, and mobile.
- **Multiple Media Types**: Support for Ebooks, Comics, and Manga with modular architecture
- **Per-Library Folders**: Each library can have multiple scanning folders for flexible organization
- **Library Visibility Control**: Admins can control which libraries are visible to each user
- **Type-Specific File Extensions**:
- **Ebooks**: `.epub`, `.pdf`, `.mobi`, `.azw`, `.azw3`, `.txt`, `.rtf`, `.doc`, `.docx`, `.lit`, `.fb2`, `.pdb`
- **Comics**: `.cbz`, `.cbr`, `.cb7`, `.cbt`, `.pdf`
- **Manga**: `.cbz`, `.cbr`, `.png`, `.jpg`, `.jpeg`, `.gif`, `.bmp`, `.webp`
- **Easy Extension**: Designed to add new media types in the future
### 🔒 Authentication & Security **📱 Multi-Library**: Organize your ebooks, comics, and manga with per-library folders and smart collections.
- **Multi-User Support**: Complete user registration and authentication system
- **JWT-Based Sessions**: Secure token-based authentication with 1-hour expiration and refresh token support
- **Role-Based Access**: Admin and user roles with granular permission control
- **Password Security**: bcrypt hashing with complex password requirements:
- Minimum 8 characters
- At least one uppercase letter (A-Z)
- At least one lowercase letter (a-z)
- At least one number (0-9)
- At least one special character (!@#$%^&*()_+-=[]{}|;':\",./<>?)
- **Account Lockout**: Automatic account lockout after 5 failed login attempts (15-minute lockout period)
- **Rate Limiting**: Built-in rate limiting on auth endpoints (10 requests/minute) to prevent brute force attacks
- **Refresh Tokens**: Secure refresh token mechanism (7-day expiration) for seamless token renewal
- **Input Validation**: Comprehensive validation including username whitespace checks, email format validation
- **Pagination Protection**: Maximum pagination limits (1000 items) to prevent DoS attacks
- **Path Validation**: Library folder paths are validated for existence and accessibility
- **Case-Insensitive Roles**: Role values automatically normalized to lowercase
- **Database Transactions**: Multi-step database operations use transaction support for data consistency
- **Standardized Error Responses**: Consistent error format across all API endpoints
### 🎨 Beautiful UI **🎨 Beautiful UI**: 11 gorgeous dark themes with responsive design that works on any device.
- **11 Dark Themes**: Tokyo Night, Dracula, Nord, Solarized Dark, Monokai, One Dark Pro, Material Dark, Catppuccin variants
- **Theme Persistence**: User theme preferences saved to database
- **Mobile-First Design**: Fully responsive interface for all devices
- **HTMX Integration**: Dynamic interactions without page reloads
### 🔄 Universal Cross-Platform Sync (NEW!) **🔒 Secure**: JWT authentication, bcrypt password hashing, rate limiting, and no passwords on devices.
- **Real-Time Progress Sync**: Reading progress syncs automatically across all your devices
- **Multi-Device Support**: KOReader, Kobo e-readers, web browsers, mobile apps
- **Format-Aware Tracking**: EPUB CFI, page numbers, percentages, character offsets
- **Highlights & Notes**: Annotations sync universally across all devices
- **Conflict Resolution**: Smart handling when same book read on multiple devices
- **Offline Support**: Queue changes when offline, sync when reconnected
- **Secure Device Auth**: Web-based device approval, no passwords on devices
- **Checkpoint Mode**: Batch sync for better battery life
- **WebSocket Updates**: Real-time broadcast of progress changes
### 📱 Media Management ---
- **Universal Media Support**: Single system for all media types with unified interface
- **Rich Metadata**: Automatic extraction of title, author, series, publisher, ISBN, tags
- **ISBN Normalization**: Automatic ISBN format normalization (removes hyphens and spaces) supporting ISBN-10 and ISBN-13 formats
- **Advanced Search**:
- Partial matching search across title, author, series, tags, and contributors
- Automatic fuzzy search fallback when no partial matches found (handles typos and misspellings)
- Real-time search results with highlighted matches
- Keyboard navigation (↑↓ arrows, Enter to select, Escape to close)
- Respects library visibility settings
- **Dynamic Sorting**: Sort your media collection by multiple fields
- Title (A-Z or Z-A)
- Author (A-Z or Z-A)
- Date added (newest or oldest first)
- Date published (newest or oldest first)
- Copyright year (newest or oldest first)
- Series order (with series number)
- Page count (shortest or longest first)
- Genre (A-Z or Z-A)
- **Advanced Filtering**: Filter media items with multiple options
- Filter by author (partial match)
- Filter by series (partial match)
- Filter by genre (exact match)
- Filter by language (English, Spanish, French, German, Japanese, Chinese, Korean, Russian, Italian, Portuguese)
- Filter by copyright year range
- Filter by items with cover images only
- Combine multiple filters with URL state management for shareable links
- **Enhanced Metadata**: New fields for better organization
- Language support for multi-lingual collections
- Edition information (2nd Edition, Revised, Collector's Edition, etc.)
- Page count for better sorting and progress calculation
- External service integration (Goodreads, OpenLibrary, Google Books IDs)
- Copyright year (distinct from publication date)
- Structured genre classification
- Subject tags (array of subjects)
- **Advanced Rating**: 5-star system with half-star precision (1-10 scale)
- **Reading Progress**: User-specific progress tracking with current page and total pages
- **Notes & Highlights**: Personal annotations and text highlighting with color customization
- **Highlight Notes**: Link highlights to detailed notes for comprehensive annotations
- **Library Statistics**: Media count and usage statistics per library
- **Background Scanning**: Automatic background scanning with configurable frequency per user
- **Watch Mode**: Real-time file system monitoring for instant library updates
## 🚀 Quick Start ## 🚀 Quick Start
### Prerequisites ### Prerequisites
- **Podman** and Podman Compose (recommended) or Docker - **Podman** (recommended) or Docker
- PostgreSQL database (handled by Podman) - **5 minutes** of your time
### Installation
### Environment Setup
1. **Create .env file**:
```bash ```bash
cp .env.example # 1. Clone the repository
git clone https://github.com/yourusername/bookhoard.git
cd bookhoard
# 2. Set up environment
cp .env.example .env
# Edit .env with your secure JWT_SECRET and DBPASS
# 3. Start the server
podman-compose up --build -d # or: docker-compose up --build -d
# 4. Open your browser
open http://localhost:8765
``` ```
2. **Edit .env** with your secure values: The first user to register automatically becomes an admin.
```bash
JWT_SECRET="your-secure-jwt-secret-key-here"
DBPASS="your-secure-database-password-here"
```
### Running the Application ---
```bash
# Using Podman (recommended)
podman-compose up --build
# Using Docker ## 📖 Key Features
docker-compose up --build
# Option 2: Direct environment variables ### Universal Cross-Platform Sync
export JWT_SECRET="your-secure-jwt-secret-key-here" - **Real-Time Progress**: Turn a page on your Kindle, see it on your phone
export DBPASS="your-secure-database-password-here" - **Format-Aware**: EPUB CFI, page numbers, percentages - all handled correctly
podman-compose up --build - **Offline Queue**: Changes sync when you reconnect, priority-processed
``` - **Conflict Resolution**: Smart handling when same book read on multiple devices
- **Book Matching**: Automatic matching using SHA-256, ISBN, UUID
- **OPDS Catalog**: Wireless book delivery to e-readers over Wi-Fi
- **Format Conversion**: On-the-fly EPUB→KEPUB for Kobo devices
Access the application at: **http://localhost:8765** ### Media Management
- **Smart Search**: Partial matching with fuzzy search fallback for typos
- **Advanced Filtering**: Filter by author, series, genre, language, year, cover images
- **Dynamic Sorting**: By title, author, date added, published date, page count, series
- **Rich Metadata**: Title, author, series, publisher, ISBN, language, edition, tags
- **5-Star Ratings**: Half-star precision (1-10 scale)
- **Notes & Highlights**: Color-coded annotations with linked notes
- **Usage Analytics**: Reading statistics, device usage, popular books
## 👤 User Management ### Smart Collections
- **Auto-Assign Rules**: Automatically add books based on genre, author, series, tags, language, publisher, year
- **Device Shelf Mappings**: Sync collections to Kobo shelves and KOReader categories
- **Test Before Creating**: Preview which books match your rules
### Role System ### Library Organization
- **Admin Users**: Can create/manage libraries, add/delete media items, manage users - **Multi-Library Support**: Ebooks, Comics, and Manga with type-specific file formats
- **Regular Users**: Can view permitted libraries, rate items, track reading progress - **Multiple Folders**: Add multiple scanning folders per library
- **Visibility Control**: Admins control which libraries each user can see
- **Background Scanning**: Auto-scan with per-user frequency settings
- **Watch Mode**: Real-time file system monitoring for instant updates
### First Admin Setup ### Security
The first user who registers automatically becomes an admin. For additional admins: - **JWT Authentication**: Short-lived access tokens (1 hour) with refresh tokens (7 days)
- **Strong Passwords**: Complexity requirements enforced (8+ chars, uppercase, lowercase, number, special)
- **Account Lockout**: 5 failed attempts = 15-minute lockout
- **Rate Limiting**: 10 requests/minute on auth endpoints
- **Input Validation**: Comprehensive validation on all inputs
- **No Passwords on Devices**: Web-based device approval with QR codes
```sql ---
-- Connect to database
docker exec -it bookhoard_db psql -U postgres -d bookhoard
-- Promote user to admin ## 📚 Documentation
UPDATE users SET role = 'admin' WHERE email = 'user@example.com';
```
## 🏛 Library Management ### For Users & Self-Hosters
- **[docs/SYNC_USER_GUIDE.md](docs/SYNC_USER_GUIDE.md)** - Understanding and using universal sync
- **[docs/devices/KOBO_SETUP.md](docs/devices/KOBO_SETUP.md)** - Kobo e-reader configuration
- **[docs/devices/KOREADER_SETUP.md](docs/devices/KOREADER_SETUP.md)** - KOReader configuration
- **[docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)** - Deployment troubleshooting
### Creating Libraries ### For Developers
1. **Admin Access**: Only administrators can create libraries - **[docs/API_REFERENCE.md](docs/API_REFERENCE.md)** - Complete API documentation
2. **Library Types**: Choose from Ebooks, Comics, or Manga - **[docs/COLLECTIONS_API.md](docs/COLLECTIONS_API.md)** - Collections API reference
3. **Multi-Folder Support**: Add multiple scanning folders per library - **[docs/api/WEBSOCKET_API.md](docs/api/WEBSOCKET_API.md)** - WebSocket protocol
4. **Folder Organization**: Organize your media collection across multiple paths - **[docs/DEVELOPMENT.md](docs/DEVELOPMENT.md)** - Development workflow
### Library Visibility Control ---
- **Admin Dashboard**: Complete interface for managing library access
- **User-Specific Control**: Admins can show/hide libraries per user
- **Personal Preferences**: Admins can also hide libraries from their own view
- **Simple Toggles**: Checkbox-based interface for easy management
### API Endpoints ## 🎯 Supported Devices
#### Notes & Highlights (All Authenticated Users) | Platform | Sync | OPDS | Status |
```bash |----------|------|------|--------|
# Media Notes | **Web Browser** | ✅ | ✅ | Full support |
GET /api/media-items/{id}/notes # Get user's notes for media | **KOReader** | ✅ | ✅ | Kindle, Kobo, PocketBook |
POST /api/media-items/{id}/notes # Create new note | **Kobo Devices** | ✅ | ✅ | Clara, Libra, Sage, etc. |
GET /api/media-items/{id}/notes/{noteId} # Get specific note | **Mobile Apps** | 🚧 | 🚧 | Coming Q2 2026 |
PUT /api/media-items/{id}/notes/{noteId} # Update note
DELETE /api/media-items/{id}/notes/{noteId} # Delete note
# Media Highlights ---
GET /api/media-items/{id}/highlights # Get user's highlights for media
POST /api/media-items/{id}/highlights # Create new highlight
GET /api/media-items/{id}/highlights/{highlightId} # Get specific highlight
PUT /api/media-items/{id}/highlights/{highlightId} # Update highlight
DELETE /api/media-items/{id}/highlights/{highlightId} # Delete highlight
```
#### Media Search (All Authenticated Users) ## 🛠 Tech Stack
```bash
# Search media items across all visible libraries
GET /api/media-items/search?q={query}
# Query Parameters: - **Backend**: Go 1.25+ with Echo framework
# q (required): Search query (minimum 2 characters) - **Database**: PostgreSQL 15+ with pgx v5
# - Searches across: title, author, series, tags, contributors - **Frontend**: HTMX + Tailwind CSS + Templ
# - Partial matching: Case-insensitive substring search - **Auth**: JWT tokens with bcrypt password hashing
# - Fuzzy fallback: Automatic when no partial matches found - **Container**: Podman (Docker compatible)
# - Results ranked by relevance
# Examples: ---
GET /api/media-items/search?q=harry%20potter # Search by title
GET /api/media-items/search?q=rowling # Search by author
GET /api/media-items/search?q=hary%20poter # Fuzzy search (handles typos)
# Response:
# - 200: Success (returns array of media items)
# - 404: No results found
# - 400: Missing or invalid query parameter
# - 401: Unauthorized
```
#### Library Management (Admin Only)
```bash
# Create new library
POST /api/libraries
{
"name": "My Comic Collection",
"description": "Digital comics and graphic novels",
"type": "comics"
}
# List all libraries
GET /api/libraries
# Add folder to library
POST /api/libraries/{library_id}/folders
{
"folder_path": "/path/to/comics"
}
# Set library visibility for user
POST /api/libraries/visibility
{
"library_id": "library-uuid",
"is_visible": true
}
# Scanner Management (Admin Only)
POST /api/scanner/scan # Manual scan of all libraries
POST /api/scanner/start # Start scheduled scanning
POST /api/scanner/stop # Stop scheduled scanning
GET /api/scanner/status/{jobId} # Get scan job status
POST /api/scanner/watch/start # Start watch mode (real-time monitoring)
POST /api/scanner/watch/stop # Stop watch mode
GET /api/scanner/watch/status # Get watch mode status
```
#### User Access
```bash
# Get user's visible libraries
GET /api/libraries/visible
# Browse media in library
GET /api/media-items?library_id={library_id}&limit=20&offset=0
# Media Progress & Ratings
GET /api/media-items/{id}/progress # Get reading progress
PUT /api/media-items/{id}/progress # Update reading progress
GET /api/media-items/{id}/rating # Get user rating
POST /api/media-items/{id}/rating # Create/update rating
PUT /api/media-items/{id}/rating # Update rating
DELETE /api/media-items/{id}/rating # Delete rating
# Scan Settings (All Authenticated Users)
GET /api/library/scan-settings # Get user's scan settings
PUT /api/library/scan-settings # Update scan settings
{
"scan_frequency_minutes": 60,
"auto_scan_enabled": true
}
# Token Management
POST /api/auth/refresh # Refresh access token
POST /api/auth/logout # Logout (revokes refresh token)
```
## 📖 Media Support
### Ebooks
- **Formats**: EPUB (full metadata), PDF, MOBI, AZW, AZW3, TXT, DOC, DOCX, LIT, FB2, PDB, RTF
- **Metadata**: Automatic extraction from EPUB files with Calibre support
- **ISBN Support**:
- Automatic normalization of ISBN-10 and ISBN-13 formats
- Removes hyphens and spaces for consistent storage
- Supports ISBN-10 with check digit X
- Database function `normalize_isbn()` handles all conversions
- **Reading**: Built-in web reader for EPUB files
- **Library Requirement**: Ebooks require an existing ebook library; graceful error handling if none exists
### Comics
- **Formats**: CBZ, CBR, CB7, CBT, PDF
- **Structure**: Archive-based organization with chapter support
- **Viewing**: Image extraction and web-based comic reader
## ⚙️ Background Processing
### Auto-Scanning
- **Scheduled Scanning**: User-configurable scan frequency (per-user settings)
- **Smart Scheduling**: Background scheduler manages scan jobs efficiently
- **Per-User Settings**: Each user can enable/disable auto-scan and set frequency
- **Resource Management**: Worker pool limits concurrent scanning operations
### Watch Mode
- **Real-Time Monitoring**: File system watcher detects new files instantly
- **Automatic Processing**: New media items processed and added to library
- **Multi-Library Support**: Watch all library folders simultaneously
- **Efficient**: Uses filesystem events for minimal resource usage
### Scan Settings API
```bash
# Get current scan settings
GET /api/library/scan-settings
# Update scan settings
PUT /api/library/scan-settings
{
"scan_frequency_minutes": 60, # How often to auto-scan (minutes)
"auto_scan_enabled": true # Enable/disable auto-scanning
}
```
### Manga
- **Formats**: CBZ, CBR (archives), PNG, JPG (image folders)
- **Structure**: Archive support with folder-based image organization
- **Viewing**: Chapter-by-chapter viewing with page navigation
## 🎨 Development
### Local Development
```bash
# Backend development
go mod tidy
go run cmd/server/main.go
# Frontend development
npm run dev
# Templates automatically recompile on changes
# Run tests
go test ./... -v
go test ./cmd/server/tests -v # Integration tests only
```
### Environment
- **Backend**: Go 1.25+ with pgx v5 for database operations
- **Frontend**: Tailwind CSS with HTMX for dynamic interactions
- **Database**: PostgreSQL 15+ with pgx v5 driver
- **Authentication**: JWT tokens with bcrypt password hashing
## 🐳 Deployment
For detailed deployment instructions, Docker configuration, and troubleshooting common issues, see **[TROUBLESHOOTING.md](TROUBLESHOOTING.md)**.
The application automatically starts background services on startup:
- **Scheduler**: Manages scheduled scanning jobs based on user settings
- **Watch Mode**: Monitors all library folders for file system changes
- **Worker Pool**: Processes scan jobs with configurable concurrency
### Database Schema
- **Multi-Library Architecture**: Libraries, library types, folders, visibility tables
- **Annotations System**: Notes, highlights with position tracking and color customization
- **Type Safety**: pgx v5 with proper error handling
- **Migration Ready**: Schema designed for easy future extensions
## 🧪 API Documentation
### Bruno Testing Collection
Complete API testing collection in `bruno/` directory:
```
bruno/
├── user/ # Authentication & profile endpoints
├── admin/ # Admin-only operations
├── library/ # Library management
├── media-items/ # Media content browsing
├── ebooks/ # Ebook-specific operations
├── notes/ # Notes API testing
├── highlights/ # Highlights API testing
├── scanner/ # Background scanning & watch mode
├── progress/ # Reading progress tracking
├── devices/ # Device management
├── conflicts/ # Sync conflict resolution
├── queue/ # Sync queue management
├── sync-koreader/ # KOReader sync protocol
├── sync-kobo/ # Kobo sync protocol
└── collection.bru # Main dashboard file
```
### Authentication Flow
1. **Register**: `POST /api/auth/register` → JWT token
2. **Login**: `POST /api/auth/login` → JWT token
3. **Protected Routes**: Use `Authorization: Bearer {token}` header
4. **Refresh Token**: `POST /api/auth/refresh` → JWT token
5. **Logout**: `POST /api/auth/logout` → Revoke refresh token
### API Reference Documentation
See [API_REFERENCE.md](API_REFERENCE.md) for complete API documentation including:
- All endpoints with request/response examples
- Bruno v3.0 test collections for all endpoints
- Error handling details
- Authentication & security features
### Device Setup Guides
- [KOBOREADER_SETUP.md](KOBOREADER_SETUP.md) - KOReader device configuration
- [KOBO_SETUP.md](KOBO_SETUP.md) - Kobo device configuration
### Universal Sync Documentation
- [UNIVERSAL_SYNC_IMPLEMENTATION_GUIDE.md](UNIVERSAL_SYNC_IMPLEMENTATION_GUIDE.md) - Complete sync architecture
### Error Handling
All API endpoints return standardized error responses:
```json
{
"error": "Error message",
"message": "Detailed error information (if available)",
"code": "Error code (if applicable)"
}
```
HTTP Status Codes:
- **400**: Bad request (validation errors)
- **401**: Unauthorized (invalid/missing token)
- **403**: Forbidden (insufficient permissions)
- **404**: Resource not found
- **429**: Too many requests (rate limit exceeded)
- **500**: Internal server error
## 🛡 Security Features
### Authentication & Authorization
- **JWT Tokens**:
- Short-lived access tokens (1-hour expiration)
- Refresh tokens (7-day expiration) for seamless session renewal
- Secure token storage and transmission
- **Password Security**:
- bcrypt hashing with cost factor 12
- Complex password requirements enforced
- Password validation on registration and updates
- **Account Protection**:
- Automatic lockout after 5 failed login attempts
- 15-minute lockout period with countdown
- IP-based and username-based attempt tracking
- **Rate Limiting**: 10 requests per minute on authentication endpoints
- **Input Validation**: Comprehensive server-side validation for all inputs
- **CSRF Protection**: Built-in with HTMX
- **Role-Based Access Control**: Admin and user roles enforced on all endpoints
### Authorization
- **Role-Based Access**: Admin vs user permissions
- **Library Visibility**: Per-user library access control
- **Admin Middleware**: Protected routes for admin operations
- **Content Security**: XSS protection and secure headers
### Database Security
- **Parameterized Queries**: SQL injection prevention with pgx v5
- **Connection Pooling**: Efficient database connection management
- **Transaction Support**: Multi-step operations wrapped in transactions
- **Environment Variables**: Secure secret management
- **Row-Level Security**: User data isolation
- **Secure Refresh Token Storage**: Encrypted token storage in database
- **Automatic Token Cleanup**: Expired tokens cleaned up periodically
- **ISBN Normalization**: Database function ensures consistent ISBN format
- **Data Validation**: Server-side validation for all inputs including ISBNs
## 🔧 Configuration
### Environment Variables
```bash
# Required
JWT_SECRET= # JWT signing secret (64-byte random string)
DBHOST=db # PostgreSQL database host
# Optional
SERVER_PORT=8765 # Application port
NODE_ENV=development # Environment mode
```
### Database Configuration
```sql
-- Library types are automatically seeded
INSERT INTO library_types (name, description, allowed_extensions) VALUES
('ebooks', 'Ebook files including EPUB, PDF, MOBI, etc.',
ARRAY['.epub', '.pdf', '.mobi', '.azw', '.azw3', '.txt', '.rtf', '.doc', '.docx', '.lit', '.fb2', '.pdb']),
('comics', 'Comic book archives and image formats',
ARRAY['.cbz', '.cbr', '.cb7', '.cbt', '.pdf']),
('manga', 'Manga files including archives and image folders',
ARRAY['.cbz', '.cbr', '.png', '.jpg', '.jpeg', '.gif', '.bmp', '.webp']);
```
## 🧪 Testing ## 🧪 Testing
### Integration Tests
Comprehensive integration test suite in `cmd/server/tests/`:
- Authentication & authorization tests
- Library management tests
- Media items operations
- Notes & highlights functionality
- ISBN normalization tests
- Background scanning & watch mode tests
- Edge cases and error handling
- Security & rate limiting tests
Run integration tests:
```bash ```bash
# Standard test run (may hit rate limits) # Run all tests
go test ./cmd/server/tests -v make test-all
# Recommended: Run with test mode enabled to avoid rate limiting # Run integration tests (with test mode)
TEST_MODE=true RATE_LIMIT_ENABLED=false go test ./cmd/server/tests -run TestIntegrationAPI -v make test-integration
# Or with increased rate limits # Run Bruno API tests
TEST_MODE=true REQUESTS_PER_MINUTE=1000 go test ./cmd/server/tests -run TestIntegrationAPI -v
```
### Test Configuration Options
The following environment variables can be used to configure test behavior:
- **TEST_MODE**: Set to `true` to enable test mode (logs additional info)
- **RATE_LIMIT_ENABLED**: Set to `false` to disable rate limiting for tests
- **REQUESTS_PER_MINUTE**: Increase rate limit (e.g., `1000`) to avoid throttling
**⚠️ WARNING**: Never disable rate limiting or enable test mode in production environments. These settings are only for integration testing.
### API Testing
```bash
# Install Bruno
npm install -g @usebruno/cli npm install -g @usebruno/cli
# Run tests
bruno run bruno run
``` ```
### Unit Tests ---
```bash
go test ./...
```
## 📊 Architecture ## 📊 Project Status
### Backend **Version**: 1.0
``` **License**: GPL-3.0
cmd/server/main.go # Application entry point **Status**: Production-ready ✅
├── internal/
│ ├── config/ # Configuration management
│ ├── database/ # Database operations (SQLC generated)
│ │ ├── queries/ # SQL queries
│ │ └── models.go # Generated models
│ ├── handlers/ # HTTP handlers
│ │ ├── auth.go # Authentication & users
│ │ ├── library.go # Library management
│ │ ├── ebook.go # Media operations
│ │ └── refresh_token.go # Token management
│ ├── services/ # Business logic
│ │ ├── library_service.go # Library service
│ │ ├── ebook_scanner.go # Media scanning
│ │ ├── worker.go # Background job processing
│ │ └── scheduler.go # Auto-scan scheduling
│ └── middleware/ # HTTP middleware
│ ├── rate_limiter.go # Rate limiting
│ ├── login_attempts.go # Account lockout
│ ├── password_validator.go # Password complexity
│ ├── error_handler.go # Error handling
│ ├── transaction.go # DB transactions
│ └── request_tracing.go # Request logging
templates/ # HTML templates with HTMX
```
### Database ---
```sql
libraries # Library definitions
library_types # Media type definitions (ebooks, comics, manga)
library_folders # Folders per library
library_visibility # User library access control
media_items # All media content (replaces ebooks table)
media_ratings # User ratings for media items
media_notes # User notes on media items
media_highlights # User highlights with optional note links
reading_progress # User reading progress
users # User accounts and profiles (includes scan settings)
refresh_tokens # JWT refresh token storage
# Database Functions ## 🤝 Contributing
normalize_isbn() # ISBN normalization function
```
## 🎯 Future Roadmap We welcome contributions! Please see [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) for guidelines.
### Recently Added Features (v1.0) ---
- ✅ **ISBN Normalization**: Automatic ISBN format standardization
- ✅ **Background Scanning**: Scheduled library scanning with per-user settings
- ✅ **Watch Mode**: Real-time file system monitoring for instant updates
- ✅ **Integration Tests**: Comprehensive test suite for all major features
- ✅ **Request Tracing**: Enhanced logging and debugging capabilities
- ✅ **Graceful Library Handling**: Better error messages when libraries don't exist
### Media Type Extensions
- **Audiobooks**: Audio file support with chapter tracking
- **Podcasts**: RSS feed integration and automatic downloading
- **Video**: Movie/TV series management with metadata
- **Music**: Album and track management with artwork
### Advanced Features
- **Mobile App**: React Native mobile application
- **API v2**: GraphQL API for efficient data fetching
- **Webhooks**: External service integrations
- **Analytics**: Usage statistics and reporting
- **Backup/Restore**: Library backup and migration tools
## 📝 Contributing
### Development Guidelines
- Follow Go best practices and effective Go
- Use pgx v5 for all database operations
- Implement proper error handling with pgx.ErrNoRows
- Write comprehensive tests for new features
- Update documentation for API changes
- Add Bruno tests for new API endpoints
- Include integration tests for complex features
### Code Style
- Follow existing code formatting
- Use meaningful variable and function names
- Add comments for complex business logic
- Ensure type safety with proper error handling
## 📄 License ## 📄 License
@@ -622,4 +157,4 @@ GPL-3.0 - See [LICENSE](LICENSE) file for details.
--- ---
**Built with ❤️ using Go, PostgreSQL, HTMX, and Tailwind CSS** **Built with ❤️ using Go, PostgreSQL, HTMX, and Tailwind CSS**
docs/API_REFERENCE.md
+327 -6
View File
@@ -12,12 +12,16 @@ Complete API documentation for Bookhoard v1.0 with Universal Cross-Platform Sync
6. [Notes & Highlights](#notes--highlights) 6. [Notes & Highlights](#notes--highlights)
7. [Ratings](#ratings) 7. [Ratings](#ratings)
8. [Device Management](#device-management) 8. [Device Management](#device-management)
9. [Sync Protocol - KOReader](#sync-protocol---koreader) 9. [Analytics](#analytics)
10. [Sync Protocol - Kobo](#sync-protocol---kobo) 10. [Book Matching & Linking](#book-matching--linking)
11. [Universal Progress](#universal-progress) 11. [Collections](#collections) → See [COLLECTIONS_API.md](COLLECTIONS_API.md)
12. [Conflicts](#conflicts) 12. [OPDS](#opds-open-publication-distribution-system)
13. [Sync Queue](#sync-queue) 13. [Sync Protocol - KOReader](#sync-protocol---koreader)
14. [WebSocket](#websocket) 14. [Sync Protocol - Kobo](#sync-protocol---kobo)
15. [Universal Progress](#universal-progress)
16. [Conflicts](#conflicts)
17. [Sync Queue](#sync-queue)
18. [WebSocket](#websocket)
## Base URL ## Base URL
@@ -735,6 +739,323 @@ DELETE /api/devices/{device_id}
Authorization: Bearer <token> Authorization: Bearer <token>
``` ```
## Analytics
### Get Reading Statistics
```http
GET /api/analytics/reading-stats?start_date={date}&end_date={date}
Authorization: Bearer <token>
```
**Query Parameters**:
- `start_date` (optional): Start date (ISO 8601 format)
- `end_date` (optional): End date (ISO 8601 format)
**Response** (200):
```json
{
"pages_read": 1250,
"books_completed": 5,
"reading_time_hours": 42.5,
"sessions_count": 28,
"average_session_minutes": 91
}
```
### Get Device Usage Statistics
```http
GET /api/analytics/device-usage
Authorization: Bearer <token>
```
**Response** (200):
```json
{
"devices": [
{
"device_id": "uuid",
"device_name": "My Kobo Clara",
"device_type": "kobo",
"sync_count": 145,
"last_sync": "2026-02-01T10:00:00Z",
"pages_synced": 890
}
]
}
```
### Get Popular Books
```http
GET /api/analytics/popular-books?limit={limit}
Authorization: Bearer <token>
```
**Query Parameters**:
- `limit` (optional): Number of results (default: 10)
**Response** (200):
```json
{
"books": [
{
"media_item_id": "uuid",
"title": "Book Title",
"author": "Author Name",
"cover_image_path": "/covers/book.jpg",
"read_count": 15,
"average_rating": 8.5
}
]
}
```
## Book Matching & Linking
### Query Books for Matching
```http
POST /api/sync/books/query
Authorization: Bearer <token>
Content-Type: application/json
{
"identifiers": ["isbn:978-0345391802", "uuid:abc-123"],
"sha256": "a1b2c3d4e5f6abc123...",
"title": "The Hobbit",
"author": "J.R.R. Tolkien",
"file_size": 2456789
}
```
**Response** (200):
```json
{
"matches": [
{
"media_item_id": "uuid-123",
"bookhoard_uuid": "uuid-123",
"confidence": 1.0,
"match_method": "uuid_match"
}
],
"action": "auto_link"
}
```
### Bulk Link Books
```http
POST /api/sync/bulk-link-books
Authorization: Bearer <token>
Content-Type: application/json
{
"links": [
{
"unlinked_book_id": "uuid-1",
"media_item_id": "uuid-2",
"confidence_score": 1.0
}
]
}
```
**Response** (200):
```json
{
"results": [
{
"unlinked_book_id": "uuid-1",
"status": "success",
"media_item_id": "uuid-2"
}
],
"total": 1,
"successful": 1,
"failed": 0
}
```
### Auto-Link Books
```http
POST /api/sync/auto-link-books
Authorization: Bearer <token>
Content-Type: application/json
{
"confidence_threshold": 0.8,
"limit": 50
}
```
**Response** (200):
```json
{
"auto_linked": 15,
"results": [
{
"unlinked_book_id": "uuid-1",
"title": "The Hobbit",
"matched_media_item_id": "uuid-2",
"confidence": 0.95,
"match_method": "sha256_match"
}
]
}
```
### Get Unlinked Book Suggestions
```http
GET /api/sync/unlinked-books/{id}/suggestions
Authorization: Bearer <token>
```
**Response** (200):
```json
{
"unlinked_book_id": "uuid-1",
"title_from_device": "The Hobbit",
"sha256": "",
"suggestions": [
{
"media_item_id": "uuid-2",
"bookhoard_uuid": "uuid-2",
"confidence": 0.95,
"match_method": "sha256_match"
}
],
"total_suggestions": 1,
"action": "auto_link"
}
```
## Collections
For complete collection management documentation, see **[COLLECTIONS_API.md](COLLECTIONS_API.md)**.
**Quick Reference**:
- `GET /api/collections` - List all collections
- `POST /api/collections` - Create new collection
- `GET /api/collections/{id}` - Get collection details
- `PUT /api/collections/{id}` - Update collection
- `DELETE /api/collections/{id}` - Delete collection
- `POST /api/collections/test-rules` - Test auto-assign rules
- `POST /api/collections/bulk-add-books` - Bulk add books to collections
- `GET /api/collections/{id}/books` - Get books in collection
**Features**:
- Auto-assign rules based on genre, author, series, tags, language, publisher, year
- Device shelf mappings (Kobo shelves, KOReader categories)
- Test rules before applying
- Bulk book operations
## OPDS (Open Publication Distribution System)
### Get Device Catalog
```http
GET /opds/devices/{deviceId}/catalog?page={page}&per_page={per_page}
```
**Query Parameters**:
- `page` (optional): Page number (default: 1)
- `per_page` (optional): Items per page (default: 50, max: 200)
**Response** (200 - OPDS 1.2 XML):
```xml
<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:opds="http://opds-spec.org/2010/"
xmlns:dc="http://purl.org/dc/elements/1.1/">
<id>urn:uuid:device-id</id>
<title>Bookhoard Library</title>
<updated>2026-02-01T12:00:00Z</updated>
<link rel="self" href="http://localhost:8765/opds/devices/kobo-id/catalog"/>
<link rel="search" href="http://localhost:8765/opds/devices/kobo-id/search"/>
<link rel="start" href="http://localhost:8765/opds/devices/kobo-id/nav"/>
<entry>
<id>urn:uuid:bookhoard-uuid-123</id>
<dc:title>The Hobbit</dc:title>
<dc:creator>J.R.R. Tolkien</dc:creator>
<updated>2026-02-01T10:00:00Z</updated>
<link href="http://localhost:8765/opds/devices/kobo-id/download/uuid-123"
type="application/epub+zip"
rel="http://opds-spec.org/acquisition/open-access"/>
<link href="http://localhost:8765/opds/devices/kobo-id/download/uuid-123?format=kepub"
type="application/vnd.kobo+xml+zip"
rel="alternate"/>
<dc:identifier id="bookhoard">uuid-123</dc:identifier>
<meta property="bookhoard:sha256">abc123...</meta>
</entry>
</feed>
```
### Download Book with Format Conversion
```http
GET /opds/devices/{deviceId}/download/{bookId}?format={format}
```
**Query Parameters**:
- `format` (optional): Book format - `epub` (default), `kepub`
**Response** (200):
- Headers:
- `Content-Type`: `application/epub+zip` or `application/vnd.kobo+xml+zip`
- `Content-Disposition`: attachment; filename="The Hobbit.epub"
- `X-Bookhoard-UUID`: uuid-123
- `X-Bookhoard-SHA256`: abc123... (original hash)
- `X-Bookhoard-KEPUB-SHA256`: xyz789... (KEPUB hash if format=kepub)
### Search OPDS Catalog
```http
GET /opds/devices/{deviceId}/search?q={query}
```
**Response** (200 - OPDS 1.2 XML with search results)
### List Available Formats
```http
GET /opds/devices/{deviceId}/formats/{bookId}
```
**Response** (200):
```json
{
"media_item_id": "uuid-123",
"formats": [
{
"format_type": "epub",
"file_path": "/path/to/book.epub",
"file_sha256": "abc123...",
"file_size_bytes": 2456789,
"mime_type": "application/epub+zip",
"available": true
},
{
"format_type": "kepub",
"file_path": "/cache/book.kepub.epub",
"file_sha256": "xyz789...",
"file_size_bytes": 2478932,
"mime_type": "application/vnd.kobo+xml+zip",
"available": true
}
]
}
```
## Sync Protocol - KOReader ## Sync Protocol - KOReader
### KOReader Progress Sync ### KOReader Progress Sync
docs/SYNC_USER_GUIDE.md
+67 -55
View File
@@ -4,10 +4,11 @@
1. [What is Universal Sync?](#what-is-universal-sync) 1. [What is Universal Sync?](#what-is-universal-sync)
2. [Supported Devices](#supported-devices) 2. [Supported Devices](#supported-devices)
3. [Getting Started](#getting-started) 3. [Getting Started](#getting-started)
4. [Device Registration](#device-registration) 4. [Device Setup](#device-setup)
5. [Sync Features](#sync-features) 5. [Using Sync Features](#using-sync-features)
6. [Troubleshooting](#troubleshooting) 6. [Book Matching](#book-matching)
7. [Best Practices](#best-practices) 7. [Managing Conflicts](#managing-conflicts)
8. [Best Practices](#best-practices)
--- ---
@@ -61,72 +62,83 @@
### Quick Start ### Quick Start
1. **Start Reading** - Open any book in the web interface or on your device 1. **Set up your first device** - See [Device Setup](#device-setup) below
2. **Turn Pages** - Progress syncs automatically 2. **Start reading** - Open any book in the web interface or on your device
3. **Switch Devices** - Pick up any other device - your progress is there! 3. **Turn pages** - Progress syncs automatically
4. **Switch devices** - Pick up any other device - your progress is there!
--- ---
## Device Registration ## Device Setup
### Step-by-Step Guide For detailed device configuration instructions, see the appropriate setup guide:
#### Option 1: Web-Based Registration (Recommended) - **[Kobo Setup Guide](devices/KOBO_SETUP.md)** - Kobo e-reader configuration
- **[KOReader Setup Guide](devices/KOREADER_SETUP.md)** - KOReader configuration
1. **On Your Device**: ### Quick Overview
- Open your reading app (KOReader/Kobo)
- Navigate to sync settings
- Find "Bookhoard Sync" or "Calibre Sync"
- Copy your device identifier (hardware ID)
2. **In Your Browser**: **Registration Process**:
- Go to your Bookhoard server 1. Register device in Bookhoard web interface (Settings → Devices)
- Log in with your account 2. Approve device via QR code or approval URL
- Navigate to **Settings → Devices** 3. Configure sync settings on your device
- Click **Register New Device** 4. Start reading - progress syncs automatically!
- Enter your device name and type
- Click **Generate Registration**
3. **Complete Registration**: **Device Management**:
- Scan the QR code OR copy the registration URL
- Visit the approval URL in your browser
- Review device details and click **Approve**
- Device will receive authentication token
4. **Configure Sync**:
- Enter the sync URL provided (e.g., `https://bookhoard.example.com/api/sync/koreader`)
- Enable auto-sync
- Set sync frequency (recommended: Every page turn)
#### Option 2: Direct Device Registration
1. **From Your Device**:
- Navigate to sync settings
- Select "Bookhoard Sync"
- Enter server URL: `https://bookhoard.example.com`
- Click "Register Device"
- Device will display registration code
2. **In Your Browser**:
- Go to `https://bookhoard.example.com/devices/approve`
- Enter registration code
- Approve the device
### Device Management
**View Your Devices**:
``` ```
Settings → Devices Settings → Devices
``` ```
**Manage Devices**: You can:
- **Rename**: Click device name → Edit - View all your registered devices
- **Disable Sync**: Toggle "Sync Enabled" - See last sync time and status
- **Remove Device**: Click "Delete" (revokes access immediately) - Disable or remove devices
- View unlinked books that need matching
- Manage file aliases
--- ---
## Sync Features ## Book Matching
### What is Book Matching?
When devices sync books, Bookhoard tries to automatically match them using:
- **SHA-256 hash** (most reliable) - Content-based fingerprint
- **ISBN** - Standard book identifier
- **UUID** - Unique identifier from EPUB metadata
- **Title + Author** (fallback) - Last resort matching
### Unlinked Books
Sometimes a book on your device can't be automatically matched to your library. This happens when:
- The book was side-loaded (not downloaded via Bookhoard)
- The file format was converted (EPUB → KEPUB)
- The metadata doesn't match exactly
### Viewing Unlinked Books
```
Settings → Devices → Select Device → View Unlinked Books
```
### Resolving Unlinked Books
1. **Auto-Link** - Automatically link books with 80%+ confidence
2. **Get Suggestions** - Review potential matches before linking
3. **Manual Link** - Choose the correct match yourself
### Book Matching Priority
1. **Bookhoard UUID** - 100% confidence
2. **OPF UUID** - 95% confidence
3. **SHA-256 hash** - 90% confidence
4. **OPF identifier** - 85% confidence
5. **ISBN/ASIN** - 80% confidence
6. **Title + Author + File Size** - 50% confidence
---
## Managing Conflicts
### Reading Progress ### Reading Progress