docs: update references for new configuration structure
- Update README.md with simplified documentation links - Update docs/API_REFERENCE.md formatting and structure - Update docs/SYNC_USER_GUIDE.md with minor improvements These updates reflect the reorganized documentation structure and new configuration approach with defaults in docker-compose.yml.
This commit is contained in:
@@ -1,620 +1,155 @@
|
||||
# 📚 Bookhoard
|
||||
|
||||
A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and Tailwind CSS featuring multiple library support, beautiful dark themes, and comprehensive media management.
|
||||
A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and Tailwind CSS featuring **universal cross-device sync**, beautiful dark themes, and comprehensive media management.
|
||||
|
||||
## ✨ Features
|
||||
## ✨ Why Bookhoard?
|
||||
|
||||
### 🏗 Multi-Library System
|
||||
- **Multiple Media Types**: Support for Ebooks, Comics, and Manga with modular architecture
|
||||
- **Per-Library Folders**: Each library can have multiple scanning folders for flexible organization
|
||||
- **Library Visibility Control**: Admins can control which libraries are visible to each user
|
||||
- **Type-Specific File Extensions**:
|
||||
- **Ebooks**: `.epub`, `.pdf`, `.mobi`, `.azw`, `.azw3`, `.txt`, `.rtf`, `.doc`, `.docx`, `.lit`, `.fb2`, `.pdb`
|
||||
- **Comics**: `.cbz`, `.cbr`, `.cb7`, `.cbt`, `.pdf`
|
||||
- **Manga**: `.cbz`, `.cbr`, `.png`, `.jpg`, `.jpeg`, `.gif`, `.bmp`, `.webp`
|
||||
- **Easy Extension**: Designed to add new media types in the future
|
||||
**🔄 Universal Sync**: Your reading progress, highlights, and notes sync automatically across all your devices - KOReader, Kobo, web, and mobile.
|
||||
|
||||
### 🔒 Authentication & Security
|
||||
- **Multi-User Support**: Complete user registration and authentication system
|
||||
- **JWT-Based Sessions**: Secure token-based authentication with 1-hour expiration and refresh token support
|
||||
- **Role-Based Access**: Admin and user roles with granular permission control
|
||||
- **Password Security**: bcrypt hashing with complex password requirements:
|
||||
- Minimum 8 characters
|
||||
- At least one uppercase letter (A-Z)
|
||||
- At least one lowercase letter (a-z)
|
||||
- At least one number (0-9)
|
||||
- At least one special character (!@#$%^&*()_+-=[]{}|;':\",./<>?)
|
||||
- **Account Lockout**: Automatic account lockout after 5 failed login attempts (15-minute lockout period)
|
||||
- **Rate Limiting**: Built-in rate limiting on auth endpoints (10 requests/minute) to prevent brute force attacks
|
||||
- **Refresh Tokens**: Secure refresh token mechanism (7-day expiration) for seamless token renewal
|
||||
- **Input Validation**: Comprehensive validation including username whitespace checks, email format validation
|
||||
- **Pagination Protection**: Maximum pagination limits (1000 items) to prevent DoS attacks
|
||||
- **Path Validation**: Library folder paths are validated for existence and accessibility
|
||||
- **Case-Insensitive Roles**: Role values automatically normalized to lowercase
|
||||
- **Database Transactions**: Multi-step database operations use transaction support for data consistency
|
||||
- **Standardized Error Responses**: Consistent error format across all API endpoints
|
||||
**📱 Multi-Library**: Organize your ebooks, comics, and manga with per-library folders and smart collections.
|
||||
|
||||
### 🎨 Beautiful UI
|
||||
- **11 Dark Themes**: Tokyo Night, Dracula, Nord, Solarized Dark, Monokai, One Dark Pro, Material Dark, Catppuccin variants
|
||||
- **Theme Persistence**: User theme preferences saved to database
|
||||
- **Mobile-First Design**: Fully responsive interface for all devices
|
||||
- **HTMX Integration**: Dynamic interactions without page reloads
|
||||
**🎨 Beautiful UI**: 11 gorgeous dark themes with responsive design that works on any device.
|
||||
|
||||
### 🔄 Universal Cross-Platform Sync (NEW!)
|
||||
- **Real-Time Progress Sync**: Reading progress syncs automatically across all your devices
|
||||
- **Multi-Device Support**: KOReader, Kobo e-readers, web browsers, mobile apps
|
||||
- **Format-Aware Tracking**: EPUB CFI, page numbers, percentages, character offsets
|
||||
- **Highlights & Notes**: Annotations sync universally across all devices
|
||||
- **Conflict Resolution**: Smart handling when same book read on multiple devices
|
||||
- **Offline Support**: Queue changes when offline, sync when reconnected
|
||||
- **Secure Device Auth**: Web-based device approval, no passwords on devices
|
||||
- **Checkpoint Mode**: Batch sync for better battery life
|
||||
- **WebSocket Updates**: Real-time broadcast of progress changes
|
||||
**🔒 Secure**: JWT authentication, bcrypt password hashing, rate limiting, and no passwords on devices.
|
||||
|
||||
### 📱 Media Management
|
||||
- **Universal Media Support**: Single system for all media types with unified interface
|
||||
- **Rich Metadata**: Automatic extraction of title, author, series, publisher, ISBN, tags
|
||||
- **ISBN Normalization**: Automatic ISBN format normalization (removes hyphens and spaces) supporting ISBN-10 and ISBN-13 formats
|
||||
- **Advanced Search**:
|
||||
- Partial matching search across title, author, series, tags, and contributors
|
||||
- Automatic fuzzy search fallback when no partial matches found (handles typos and misspellings)
|
||||
- Real-time search results with highlighted matches
|
||||
- Keyboard navigation (↑↓ arrows, Enter to select, Escape to close)
|
||||
- Respects library visibility settings
|
||||
- **Dynamic Sorting**: Sort your media collection by multiple fields
|
||||
- Title (A-Z or Z-A)
|
||||
- Author (A-Z or Z-A)
|
||||
- Date added (newest or oldest first)
|
||||
- Date published (newest or oldest first)
|
||||
- Copyright year (newest or oldest first)
|
||||
- Series order (with series number)
|
||||
- Page count (shortest or longest first)
|
||||
- Genre (A-Z or Z-A)
|
||||
- **Advanced Filtering**: Filter media items with multiple options
|
||||
- Filter by author (partial match)
|
||||
- Filter by series (partial match)
|
||||
- Filter by genre (exact match)
|
||||
- Filter by language (English, Spanish, French, German, Japanese, Chinese, Korean, Russian, Italian, Portuguese)
|
||||
- Filter by copyright year range
|
||||
- Filter by items with cover images only
|
||||
- Combine multiple filters with URL state management for shareable links
|
||||
- **Enhanced Metadata**: New fields for better organization
|
||||
- Language support for multi-lingual collections
|
||||
- Edition information (2nd Edition, Revised, Collector's Edition, etc.)
|
||||
- Page count for better sorting and progress calculation
|
||||
- External service integration (Goodreads, OpenLibrary, Google Books IDs)
|
||||
- Copyright year (distinct from publication date)
|
||||
- Structured genre classification
|
||||
- Subject tags (array of subjects)
|
||||
- **Advanced Rating**: 5-star system with half-star precision (1-10 scale)
|
||||
- **Reading Progress**: User-specific progress tracking with current page and total pages
|
||||
- **Notes & Highlights**: Personal annotations and text highlighting with color customization
|
||||
- **Highlight Notes**: Link highlights to detailed notes for comprehensive annotations
|
||||
- **Library Statistics**: Media count and usage statistics per library
|
||||
- **Background Scanning**: Automatic background scanning with configurable frequency per user
|
||||
- **Watch Mode**: Real-time file system monitoring for instant library updates
|
||||
---
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Prerequisites
|
||||
- **Podman** and Podman Compose (recommended) or Docker
|
||||
- PostgreSQL database (handled by Podman)
|
||||
- **Podman** (recommended) or Docker
|
||||
- **5 minutes** of your time
|
||||
|
||||
### Installation
|
||||
|
||||
### Environment Setup
|
||||
1. **Create .env file**:
|
||||
```bash
|
||||
cp .env.example
|
||||
# 1. Clone the repository
|
||||
git clone https://github.com/yourusername/bookhoard.git
|
||||
cd bookhoard
|
||||
|
||||
# 2. Set up environment
|
||||
cp .env.example .env
|
||||
# Edit .env with your secure JWT_SECRET and DBPASS
|
||||
|
||||
# 3. Start the server
|
||||
podman-compose up --build -d # or: docker-compose up --build -d
|
||||
|
||||
# 4. Open your browser
|
||||
open http://localhost:8765
|
||||
```
|
||||
|
||||
2. **Edit .env** with your secure values:
|
||||
```bash
|
||||
JWT_SECRET="your-secure-jwt-secret-key-here"
|
||||
DBPASS="your-secure-database-password-here"
|
||||
```
|
||||
The first user to register automatically becomes an admin.
|
||||
|
||||
### Running the Application
|
||||
```bash
|
||||
# Using Podman (recommended)
|
||||
podman-compose up --build
|
||||
---
|
||||
|
||||
# Using Docker
|
||||
docker-compose up --build
|
||||
## 📖 Key Features
|
||||
|
||||
# Option 2: Direct environment variables
|
||||
export JWT_SECRET="your-secure-jwt-secret-key-here"
|
||||
export DBPASS="your-secure-database-password-here"
|
||||
podman-compose up --build
|
||||
```
|
||||
### Universal Cross-Platform Sync
|
||||
- **Real-Time Progress**: Turn a page on your Kindle, see it on your phone
|
||||
- **Format-Aware**: EPUB CFI, page numbers, percentages - all handled correctly
|
||||
- **Offline Queue**: Changes sync when you reconnect, priority-processed
|
||||
- **Conflict Resolution**: Smart handling when same book read on multiple devices
|
||||
- **Book Matching**: Automatic matching using SHA-256, ISBN, UUID
|
||||
- **OPDS Catalog**: Wireless book delivery to e-readers over Wi-Fi
|
||||
- **Format Conversion**: On-the-fly EPUB→KEPUB for Kobo devices
|
||||
|
||||
Access the application at: **http://localhost:8765**
|
||||
### Media Management
|
||||
- **Smart Search**: Partial matching with fuzzy search fallback for typos
|
||||
- **Advanced Filtering**: Filter by author, series, genre, language, year, cover images
|
||||
- **Dynamic Sorting**: By title, author, date added, published date, page count, series
|
||||
- **Rich Metadata**: Title, author, series, publisher, ISBN, language, edition, tags
|
||||
- **5-Star Ratings**: Half-star precision (1-10 scale)
|
||||
- **Notes & Highlights**: Color-coded annotations with linked notes
|
||||
- **Usage Analytics**: Reading statistics, device usage, popular books
|
||||
|
||||
## 👤 User Management
|
||||
### Smart Collections
|
||||
- **Auto-Assign Rules**: Automatically add books based on genre, author, series, tags, language, publisher, year
|
||||
- **Device Shelf Mappings**: Sync collections to Kobo shelves and KOReader categories
|
||||
- **Test Before Creating**: Preview which books match your rules
|
||||
|
||||
### Role System
|
||||
- **Admin Users**: Can create/manage libraries, add/delete media items, manage users
|
||||
- **Regular Users**: Can view permitted libraries, rate items, track reading progress
|
||||
### Library Organization
|
||||
- **Multi-Library Support**: Ebooks, Comics, and Manga with type-specific file formats
|
||||
- **Multiple Folders**: Add multiple scanning folders per library
|
||||
- **Visibility Control**: Admins control which libraries each user can see
|
||||
- **Background Scanning**: Auto-scan with per-user frequency settings
|
||||
- **Watch Mode**: Real-time file system monitoring for instant updates
|
||||
|
||||
### First Admin Setup
|
||||
The first user who registers automatically becomes an admin. For additional admins:
|
||||
### Security
|
||||
- **JWT Authentication**: Short-lived access tokens (1 hour) with refresh tokens (7 days)
|
||||
- **Strong Passwords**: Complexity requirements enforced (8+ chars, uppercase, lowercase, number, special)
|
||||
- **Account Lockout**: 5 failed attempts = 15-minute lockout
|
||||
- **Rate Limiting**: 10 requests/minute on auth endpoints
|
||||
- **Input Validation**: Comprehensive validation on all inputs
|
||||
- **No Passwords on Devices**: Web-based device approval with QR codes
|
||||
|
||||
```sql
|
||||
-- Connect to database
|
||||
docker exec -it bookhoard_db psql -U postgres -d bookhoard
|
||||
---
|
||||
|
||||
-- Promote user to admin
|
||||
UPDATE users SET role = 'admin' WHERE email = 'user@example.com';
|
||||
```
|
||||
## 📚 Documentation
|
||||
|
||||
## 🏛 Library Management
|
||||
### For Users & Self-Hosters
|
||||
- **[docs/SYNC_USER_GUIDE.md](docs/SYNC_USER_GUIDE.md)** - Understanding and using universal sync
|
||||
- **[docs/devices/KOBO_SETUP.md](docs/devices/KOBO_SETUP.md)** - Kobo e-reader configuration
|
||||
- **[docs/devices/KOREADER_SETUP.md](docs/devices/KOREADER_SETUP.md)** - KOReader configuration
|
||||
- **[docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)** - Deployment troubleshooting
|
||||
|
||||
### Creating Libraries
|
||||
1. **Admin Access**: Only administrators can create libraries
|
||||
2. **Library Types**: Choose from Ebooks, Comics, or Manga
|
||||
3. **Multi-Folder Support**: Add multiple scanning folders per library
|
||||
4. **Folder Organization**: Organize your media collection across multiple paths
|
||||
### For Developers
|
||||
- **[docs/API_REFERENCE.md](docs/API_REFERENCE.md)** - Complete API documentation
|
||||
- **[docs/COLLECTIONS_API.md](docs/COLLECTIONS_API.md)** - Collections API reference
|
||||
- **[docs/api/WEBSOCKET_API.md](docs/api/WEBSOCKET_API.md)** - WebSocket protocol
|
||||
- **[docs/DEVELOPMENT.md](docs/DEVELOPMENT.md)** - Development workflow
|
||||
|
||||
### Library Visibility Control
|
||||
- **Admin Dashboard**: Complete interface for managing library access
|
||||
- **User-Specific Control**: Admins can show/hide libraries per user
|
||||
- **Personal Preferences**: Admins can also hide libraries from their own view
|
||||
- **Simple Toggles**: Checkbox-based interface for easy management
|
||||
---
|
||||
|
||||
### API Endpoints
|
||||
## 🎯 Supported Devices
|
||||
|
||||
#### Notes & Highlights (All Authenticated Users)
|
||||
```bash
|
||||
# Media Notes
|
||||
GET /api/media-items/{id}/notes # Get user's notes for media
|
||||
POST /api/media-items/{id}/notes # Create new note
|
||||
GET /api/media-items/{id}/notes/{noteId} # Get specific note
|
||||
PUT /api/media-items/{id}/notes/{noteId} # Update note
|
||||
DELETE /api/media-items/{id}/notes/{noteId} # Delete note
|
||||
| Platform | Sync | OPDS | Status |
|
||||
|----------|------|------|--------|
|
||||
| **Web Browser** | ✅ | ✅ | Full support |
|
||||
| **KOReader** | ✅ | ✅ | Kindle, Kobo, PocketBook |
|
||||
| **Kobo Devices** | ✅ | ✅ | Clara, Libra, Sage, etc. |
|
||||
| **Mobile Apps** | 🚧 | 🚧 | Coming Q2 2026 |
|
||||
|
||||
# Media Highlights
|
||||
GET /api/media-items/{id}/highlights # Get user's highlights for media
|
||||
POST /api/media-items/{id}/highlights # Create new highlight
|
||||
GET /api/media-items/{id}/highlights/{highlightId} # Get specific highlight
|
||||
PUT /api/media-items/{id}/highlights/{highlightId} # Update highlight
|
||||
DELETE /api/media-items/{id}/highlights/{highlightId} # Delete highlight
|
||||
```
|
||||
---
|
||||
|
||||
#### Media Search (All Authenticated Users)
|
||||
```bash
|
||||
# Search media items across all visible libraries
|
||||
GET /api/media-items/search?q={query}
|
||||
## 🛠 Tech Stack
|
||||
|
||||
# Query Parameters:
|
||||
# q (required): Search query (minimum 2 characters)
|
||||
# - Searches across: title, author, series, tags, contributors
|
||||
# - Partial matching: Case-insensitive substring search
|
||||
# - Fuzzy fallback: Automatic when no partial matches found
|
||||
# - Results ranked by relevance
|
||||
- **Backend**: Go 1.25+ with Echo framework
|
||||
- **Database**: PostgreSQL 15+ with pgx v5
|
||||
- **Frontend**: HTMX + Tailwind CSS + Templ
|
||||
- **Auth**: JWT tokens with bcrypt password hashing
|
||||
- **Container**: Podman (Docker compatible)
|
||||
|
||||
# Examples:
|
||||
GET /api/media-items/search?q=harry%20potter # Search by title
|
||||
GET /api/media-items/search?q=rowling # Search by author
|
||||
GET /api/media-items/search?q=hary%20poter # Fuzzy search (handles typos)
|
||||
|
||||
# Response:
|
||||
# - 200: Success (returns array of media items)
|
||||
# - 404: No results found
|
||||
# - 400: Missing or invalid query parameter
|
||||
# - 401: Unauthorized
|
||||
```
|
||||
|
||||
#### Library Management (Admin Only)
|
||||
```bash
|
||||
# Create new library
|
||||
POST /api/libraries
|
||||
{
|
||||
"name": "My Comic Collection",
|
||||
"description": "Digital comics and graphic novels",
|
||||
"type": "comics"
|
||||
}
|
||||
|
||||
# List all libraries
|
||||
GET /api/libraries
|
||||
|
||||
# Add folder to library
|
||||
POST /api/libraries/{library_id}/folders
|
||||
{
|
||||
"folder_path": "/path/to/comics"
|
||||
}
|
||||
|
||||
# Set library visibility for user
|
||||
POST /api/libraries/visibility
|
||||
{
|
||||
"library_id": "library-uuid",
|
||||
"is_visible": true
|
||||
}
|
||||
|
||||
# Scanner Management (Admin Only)
|
||||
POST /api/scanner/scan # Manual scan of all libraries
|
||||
POST /api/scanner/start # Start scheduled scanning
|
||||
POST /api/scanner/stop # Stop scheduled scanning
|
||||
GET /api/scanner/status/{jobId} # Get scan job status
|
||||
POST /api/scanner/watch/start # Start watch mode (real-time monitoring)
|
||||
POST /api/scanner/watch/stop # Stop watch mode
|
||||
GET /api/scanner/watch/status # Get watch mode status
|
||||
```
|
||||
|
||||
#### User Access
|
||||
```bash
|
||||
# Get user's visible libraries
|
||||
GET /api/libraries/visible
|
||||
|
||||
# Browse media in library
|
||||
GET /api/media-items?library_id={library_id}&limit=20&offset=0
|
||||
|
||||
# Media Progress & Ratings
|
||||
GET /api/media-items/{id}/progress # Get reading progress
|
||||
PUT /api/media-items/{id}/progress # Update reading progress
|
||||
GET /api/media-items/{id}/rating # Get user rating
|
||||
POST /api/media-items/{id}/rating # Create/update rating
|
||||
PUT /api/media-items/{id}/rating # Update rating
|
||||
DELETE /api/media-items/{id}/rating # Delete rating
|
||||
|
||||
# Scan Settings (All Authenticated Users)
|
||||
GET /api/library/scan-settings # Get user's scan settings
|
||||
PUT /api/library/scan-settings # Update scan settings
|
||||
{
|
||||
"scan_frequency_minutes": 60,
|
||||
"auto_scan_enabled": true
|
||||
}
|
||||
|
||||
# Token Management
|
||||
POST /api/auth/refresh # Refresh access token
|
||||
POST /api/auth/logout # Logout (revokes refresh token)
|
||||
```
|
||||
|
||||
## 📖 Media Support
|
||||
|
||||
### Ebooks
|
||||
- **Formats**: EPUB (full metadata), PDF, MOBI, AZW, AZW3, TXT, DOC, DOCX, LIT, FB2, PDB, RTF
|
||||
- **Metadata**: Automatic extraction from EPUB files with Calibre support
|
||||
- **ISBN Support**:
|
||||
- Automatic normalization of ISBN-10 and ISBN-13 formats
|
||||
- Removes hyphens and spaces for consistent storage
|
||||
- Supports ISBN-10 with check digit X
|
||||
- Database function `normalize_isbn()` handles all conversions
|
||||
- **Reading**: Built-in web reader for EPUB files
|
||||
- **Library Requirement**: Ebooks require an existing ebook library; graceful error handling if none exists
|
||||
|
||||
### Comics
|
||||
- **Formats**: CBZ, CBR, CB7, CBT, PDF
|
||||
- **Structure**: Archive-based organization with chapter support
|
||||
- **Viewing**: Image extraction and web-based comic reader
|
||||
|
||||
## ⚙️ Background Processing
|
||||
|
||||
### Auto-Scanning
|
||||
- **Scheduled Scanning**: User-configurable scan frequency (per-user settings)
|
||||
- **Smart Scheduling**: Background scheduler manages scan jobs efficiently
|
||||
- **Per-User Settings**: Each user can enable/disable auto-scan and set frequency
|
||||
- **Resource Management**: Worker pool limits concurrent scanning operations
|
||||
|
||||
### Watch Mode
|
||||
- **Real-Time Monitoring**: File system watcher detects new files instantly
|
||||
- **Automatic Processing**: New media items processed and added to library
|
||||
- **Multi-Library Support**: Watch all library folders simultaneously
|
||||
- **Efficient**: Uses filesystem events for minimal resource usage
|
||||
|
||||
### Scan Settings API
|
||||
```bash
|
||||
# Get current scan settings
|
||||
GET /api/library/scan-settings
|
||||
|
||||
# Update scan settings
|
||||
PUT /api/library/scan-settings
|
||||
{
|
||||
"scan_frequency_minutes": 60, # How often to auto-scan (minutes)
|
||||
"auto_scan_enabled": true # Enable/disable auto-scanning
|
||||
}
|
||||
```
|
||||
|
||||
### Manga
|
||||
- **Formats**: CBZ, CBR (archives), PNG, JPG (image folders)
|
||||
- **Structure**: Archive support with folder-based image organization
|
||||
- **Viewing**: Chapter-by-chapter viewing with page navigation
|
||||
|
||||
## 🎨 Development
|
||||
|
||||
### Local Development
|
||||
```bash
|
||||
# Backend development
|
||||
go mod tidy
|
||||
go run cmd/server/main.go
|
||||
|
||||
# Frontend development
|
||||
npm run dev
|
||||
# Templates automatically recompile on changes
|
||||
|
||||
# Run tests
|
||||
go test ./... -v
|
||||
go test ./cmd/server/tests -v # Integration tests only
|
||||
```
|
||||
|
||||
### Environment
|
||||
- **Backend**: Go 1.25+ with pgx v5 for database operations
|
||||
- **Frontend**: Tailwind CSS with HTMX for dynamic interactions
|
||||
- **Database**: PostgreSQL 15+ with pgx v5 driver
|
||||
- **Authentication**: JWT tokens with bcrypt password hashing
|
||||
|
||||
## 🐳 Deployment
|
||||
|
||||
For detailed deployment instructions, Docker configuration, and troubleshooting common issues, see **[TROUBLESHOOTING.md](TROUBLESHOOTING.md)**.
|
||||
|
||||
The application automatically starts background services on startup:
|
||||
- **Scheduler**: Manages scheduled scanning jobs based on user settings
|
||||
- **Watch Mode**: Monitors all library folders for file system changes
|
||||
- **Worker Pool**: Processes scan jobs with configurable concurrency
|
||||
|
||||
### Database Schema
|
||||
- **Multi-Library Architecture**: Libraries, library types, folders, visibility tables
|
||||
- **Annotations System**: Notes, highlights with position tracking and color customization
|
||||
- **Type Safety**: pgx v5 with proper error handling
|
||||
- **Migration Ready**: Schema designed for easy future extensions
|
||||
|
||||
## 🧪 API Documentation
|
||||
|
||||
### Bruno Testing Collection
|
||||
Complete API testing collection in `bruno/` directory:
|
||||
|
||||
```
|
||||
bruno/
|
||||
├── user/ # Authentication & profile endpoints
|
||||
├── admin/ # Admin-only operations
|
||||
├── library/ # Library management
|
||||
├── media-items/ # Media content browsing
|
||||
├── ebooks/ # Ebook-specific operations
|
||||
├── notes/ # Notes API testing
|
||||
├── highlights/ # Highlights API testing
|
||||
├── scanner/ # Background scanning & watch mode
|
||||
├── progress/ # Reading progress tracking
|
||||
├── devices/ # Device management
|
||||
├── conflicts/ # Sync conflict resolution
|
||||
├── queue/ # Sync queue management
|
||||
├── sync-koreader/ # KOReader sync protocol
|
||||
├── sync-kobo/ # Kobo sync protocol
|
||||
└── collection.bru # Main dashboard file
|
||||
```
|
||||
|
||||
### Authentication Flow
|
||||
1. **Register**: `POST /api/auth/register` → JWT token
|
||||
2. **Login**: `POST /api/auth/login` → JWT token
|
||||
3. **Protected Routes**: Use `Authorization: Bearer {token}` header
|
||||
4. **Refresh Token**: `POST /api/auth/refresh` → JWT token
|
||||
5. **Logout**: `POST /api/auth/logout` → Revoke refresh token
|
||||
|
||||
### API Reference Documentation
|
||||
See [API_REFERENCE.md](API_REFERENCE.md) for complete API documentation including:
|
||||
- All endpoints with request/response examples
|
||||
- Bruno v3.0 test collections for all endpoints
|
||||
- Error handling details
|
||||
- Authentication & security features
|
||||
|
||||
### Device Setup Guides
|
||||
- [KOBOREADER_SETUP.md](KOBOREADER_SETUP.md) - KOReader device configuration
|
||||
- [KOBO_SETUP.md](KOBO_SETUP.md) - Kobo device configuration
|
||||
|
||||
### Universal Sync Documentation
|
||||
- [UNIVERSAL_SYNC_IMPLEMENTATION_GUIDE.md](UNIVERSAL_SYNC_IMPLEMENTATION_GUIDE.md) - Complete sync architecture
|
||||
|
||||
### Error Handling
|
||||
All API endpoints return standardized error responses:
|
||||
```json
|
||||
{
|
||||
"error": "Error message",
|
||||
"message": "Detailed error information (if available)",
|
||||
"code": "Error code (if applicable)"
|
||||
}
|
||||
```
|
||||
|
||||
HTTP Status Codes:
|
||||
- **400**: Bad request (validation errors)
|
||||
- **401**: Unauthorized (invalid/missing token)
|
||||
- **403**: Forbidden (insufficient permissions)
|
||||
- **404**: Resource not found
|
||||
- **429**: Too many requests (rate limit exceeded)
|
||||
- **500**: Internal server error
|
||||
|
||||
## 🛡 Security Features
|
||||
|
||||
### Authentication & Authorization
|
||||
- **JWT Tokens**:
|
||||
- Short-lived access tokens (1-hour expiration)
|
||||
- Refresh tokens (7-day expiration) for seamless session renewal
|
||||
- Secure token storage and transmission
|
||||
- **Password Security**:
|
||||
- bcrypt hashing with cost factor 12
|
||||
- Complex password requirements enforced
|
||||
- Password validation on registration and updates
|
||||
- **Account Protection**:
|
||||
- Automatic lockout after 5 failed login attempts
|
||||
- 15-minute lockout period with countdown
|
||||
- IP-based and username-based attempt tracking
|
||||
- **Rate Limiting**: 10 requests per minute on authentication endpoints
|
||||
- **Input Validation**: Comprehensive server-side validation for all inputs
|
||||
- **CSRF Protection**: Built-in with HTMX
|
||||
- **Role-Based Access Control**: Admin and user roles enforced on all endpoints
|
||||
|
||||
### Authorization
|
||||
- **Role-Based Access**: Admin vs user permissions
|
||||
- **Library Visibility**: Per-user library access control
|
||||
- **Admin Middleware**: Protected routes for admin operations
|
||||
- **Content Security**: XSS protection and secure headers
|
||||
|
||||
### Database Security
|
||||
- **Parameterized Queries**: SQL injection prevention with pgx v5
|
||||
- **Connection Pooling**: Efficient database connection management
|
||||
- **Transaction Support**: Multi-step operations wrapped in transactions
|
||||
- **Environment Variables**: Secure secret management
|
||||
- **Row-Level Security**: User data isolation
|
||||
- **Secure Refresh Token Storage**: Encrypted token storage in database
|
||||
- **Automatic Token Cleanup**: Expired tokens cleaned up periodically
|
||||
- **ISBN Normalization**: Database function ensures consistent ISBN format
|
||||
- **Data Validation**: Server-side validation for all inputs including ISBNs
|
||||
|
||||
## 🔧 Configuration
|
||||
|
||||
### Environment Variables
|
||||
```bash
|
||||
# Required
|
||||
JWT_SECRET= # JWT signing secret (64-byte random string)
|
||||
DBHOST=db # PostgreSQL database host
|
||||
|
||||
# Optional
|
||||
SERVER_PORT=8765 # Application port
|
||||
NODE_ENV=development # Environment mode
|
||||
```
|
||||
|
||||
### Database Configuration
|
||||
```sql
|
||||
-- Library types are automatically seeded
|
||||
INSERT INTO library_types (name, description, allowed_extensions) VALUES
|
||||
('ebooks', 'Ebook files including EPUB, PDF, MOBI, etc.',
|
||||
ARRAY['.epub', '.pdf', '.mobi', '.azw', '.azw3', '.txt', '.rtf', '.doc', '.docx', '.lit', '.fb2', '.pdb']),
|
||||
('comics', 'Comic book archives and image formats',
|
||||
ARRAY['.cbz', '.cbr', '.cb7', '.cbt', '.pdf']),
|
||||
('manga', 'Manga files including archives and image folders',
|
||||
ARRAY['.cbz', '.cbr', '.png', '.jpg', '.jpeg', '.gif', '.bmp', '.webp']);
|
||||
```
|
||||
---
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
### Integration Tests
|
||||
Comprehensive integration test suite in `cmd/server/tests/`:
|
||||
- Authentication & authorization tests
|
||||
- Library management tests
|
||||
- Media items operations
|
||||
- Notes & highlights functionality
|
||||
- ISBN normalization tests
|
||||
- Background scanning & watch mode tests
|
||||
- Edge cases and error handling
|
||||
- Security & rate limiting tests
|
||||
|
||||
Run integration tests:
|
||||
```bash
|
||||
# Standard test run (may hit rate limits)
|
||||
go test ./cmd/server/tests -v
|
||||
# Run all tests
|
||||
make test-all
|
||||
|
||||
# Recommended: Run with test mode enabled to avoid rate limiting
|
||||
TEST_MODE=true RATE_LIMIT_ENABLED=false go test ./cmd/server/tests -run TestIntegrationAPI -v
|
||||
# Run integration tests (with test mode)
|
||||
make test-integration
|
||||
|
||||
# Or with increased rate limits
|
||||
TEST_MODE=true REQUESTS_PER_MINUTE=1000 go test ./cmd/server/tests -run TestIntegrationAPI -v
|
||||
```
|
||||
|
||||
### Test Configuration Options
|
||||
The following environment variables can be used to configure test behavior:
|
||||
|
||||
- **TEST_MODE**: Set to `true` to enable test mode (logs additional info)
|
||||
- **RATE_LIMIT_ENABLED**: Set to `false` to disable rate limiting for tests
|
||||
- **REQUESTS_PER_MINUTE**: Increase rate limit (e.g., `1000`) to avoid throttling
|
||||
|
||||
**⚠️ WARNING**: Never disable rate limiting or enable test mode in production environments. These settings are only for integration testing.
|
||||
|
||||
### API Testing
|
||||
```bash
|
||||
# Install Bruno
|
||||
# Run Bruno API tests
|
||||
npm install -g @usebruno/cli
|
||||
|
||||
# Run tests
|
||||
bruno run
|
||||
```
|
||||
|
||||
### Unit Tests
|
||||
```bash
|
||||
go test ./...
|
||||
```
|
||||
---
|
||||
|
||||
## 📊 Architecture
|
||||
## 📊 Project Status
|
||||
|
||||
### Backend
|
||||
```
|
||||
cmd/server/main.go # Application entry point
|
||||
├── internal/
|
||||
│ ├── config/ # Configuration management
|
||||
│ ├── database/ # Database operations (SQLC generated)
|
||||
│ │ ├── queries/ # SQL queries
|
||||
│ │ └── models.go # Generated models
|
||||
│ ├── handlers/ # HTTP handlers
|
||||
│ │ ├── auth.go # Authentication & users
|
||||
│ │ ├── library.go # Library management
|
||||
│ │ ├── ebook.go # Media operations
|
||||
│ │ └── refresh_token.go # Token management
|
||||
│ ├── services/ # Business logic
|
||||
│ │ ├── library_service.go # Library service
|
||||
│ │ ├── ebook_scanner.go # Media scanning
|
||||
│ │ ├── worker.go # Background job processing
|
||||
│ │ └── scheduler.go # Auto-scan scheduling
|
||||
│ └── middleware/ # HTTP middleware
|
||||
│ ├── rate_limiter.go # Rate limiting
|
||||
│ ├── login_attempts.go # Account lockout
|
||||
│ ├── password_validator.go # Password complexity
|
||||
│ ├── error_handler.go # Error handling
|
||||
│ ├── transaction.go # DB transactions
|
||||
│ └── request_tracing.go # Request logging
|
||||
templates/ # HTML templates with HTMX
|
||||
```
|
||||
**Version**: 1.0
|
||||
**License**: GPL-3.0
|
||||
**Status**: Production-ready ✅
|
||||
|
||||
### Database
|
||||
```sql
|
||||
libraries # Library definitions
|
||||
library_types # Media type definitions (ebooks, comics, manga)
|
||||
library_folders # Folders per library
|
||||
library_visibility # User library access control
|
||||
media_items # All media content (replaces ebooks table)
|
||||
media_ratings # User ratings for media items
|
||||
media_notes # User notes on media items
|
||||
media_highlights # User highlights with optional note links
|
||||
reading_progress # User reading progress
|
||||
users # User accounts and profiles (includes scan settings)
|
||||
refresh_tokens # JWT refresh token storage
|
||||
---
|
||||
|
||||
# Database Functions
|
||||
normalize_isbn() # ISBN normalization function
|
||||
```
|
||||
## 🤝 Contributing
|
||||
|
||||
## 🎯 Future Roadmap
|
||||
We welcome contributions! Please see [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) for guidelines.
|
||||
|
||||
### Recently Added Features (v1.0)
|
||||
- ✅ **ISBN Normalization**: Automatic ISBN format standardization
|
||||
- ✅ **Background Scanning**: Scheduled library scanning with per-user settings
|
||||
- ✅ **Watch Mode**: Real-time file system monitoring for instant updates
|
||||
- ✅ **Integration Tests**: Comprehensive test suite for all major features
|
||||
- ✅ **Request Tracing**: Enhanced logging and debugging capabilities
|
||||
- ✅ **Graceful Library Handling**: Better error messages when libraries don't exist
|
||||
|
||||
### Media Type Extensions
|
||||
- **Audiobooks**: Audio file support with chapter tracking
|
||||
- **Podcasts**: RSS feed integration and automatic downloading
|
||||
- **Video**: Movie/TV series management with metadata
|
||||
- **Music**: Album and track management with artwork
|
||||
|
||||
### Advanced Features
|
||||
- **Mobile App**: React Native mobile application
|
||||
- **API v2**: GraphQL API for efficient data fetching
|
||||
- **Webhooks**: External service integrations
|
||||
- **Analytics**: Usage statistics and reporting
|
||||
- **Backup/Restore**: Library backup and migration tools
|
||||
|
||||
## 📝 Contributing
|
||||
|
||||
### Development Guidelines
|
||||
- Follow Go best practices and effective Go
|
||||
- Use pgx v5 for all database operations
|
||||
- Implement proper error handling with pgx.ErrNoRows
|
||||
- Write comprehensive tests for new features
|
||||
- Update documentation for API changes
|
||||
- Add Bruno tests for new API endpoints
|
||||
- Include integration tests for complex features
|
||||
|
||||
### Code Style
|
||||
- Follow existing code formatting
|
||||
- Use meaningful variable and function names
|
||||
- Add comments for complex business logic
|
||||
- Ensure type safety with proper error handling
|
||||
---
|
||||
|
||||
## 📄 License
|
||||
|
||||
@@ -622,4 +157,4 @@ GPL-3.0 - See [LICENSE](LICENSE) file for details.
|
||||
|
||||
---
|
||||
|
||||
**Built with ❤️ using Go, PostgreSQL, HTMX, and Tailwind CSS**
|
||||
**Built with ❤️ using Go, PostgreSQL, HTMX, and Tailwind CSS**
|
||||
|
||||
Reference in New Issue
Block a user