docs(readme): update README with current project features
- Add ISBN normalization documentation - Document background scanning and watch mode features - Add scan settings API endpoints - Include integration testing section - Update architecture section with new services - Document auto-starting services - Add recently added features section - Update testing documentation with integration tests - Enhance security section with ISBN validation
This commit is contained in:
@@ -43,11 +43,14 @@ A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and T
|
||||
### 📱 Media Management
|
||||
- **Universal Media Support**: Single system for all media types with unified interface
|
||||
- **Rich Metadata**: Automatic extraction of title, author, series, publisher, ISBN, tags
|
||||
- **ISBN Normalization**: Automatic ISBN format normalization (removes hyphens and spaces) supporting ISBN-10 and ISBN-13 formats
|
||||
- **Advanced Rating**: 5-star system with half-star precision (1-10 scale)
|
||||
- **Reading Progress**: User-specific progress tracking with current page and total pages
|
||||
- **Notes & Highlights**: Personal annotations and text highlighting with color customization
|
||||
- **Highlight Notes**: Link highlights to detailed notes for comprehensive annotations
|
||||
- **Library Statistics**: Media count and usage statistics per library
|
||||
- **Background Scanning**: Automatic background scanning with configurable frequency per user
|
||||
- **Watch Mode**: Real-time file system monitoring for instant library updates
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
@@ -166,6 +169,15 @@ POST /api/libraries/visibility
|
||||
"library_id": "library-uuid",
|
||||
"is_visible": true
|
||||
}
|
||||
|
||||
# Scanner Management (Admin Only)
|
||||
POST /api/scanner/scan # Manual scan of all libraries
|
||||
POST /api/scanner/start # Start scheduled scanning
|
||||
POST /api/scanner/stop # Stop scheduled scanning
|
||||
GET /api/scanner/status/{jobId} # Get scan job status
|
||||
POST /api/scanner/watch/start # Start watch mode (real-time monitoring)
|
||||
POST /api/scanner/watch/stop # Stop watch mode
|
||||
GET /api/scanner/watch/status # Get watch mode status
|
||||
```
|
||||
|
||||
#### User Access
|
||||
@@ -184,6 +196,14 @@ POST /api/media-items/{id}/rating # Create/update rating
|
||||
PUT /api/media-items/{id}/rating # Update rating
|
||||
DELETE /api/media-items/{id}/rating # Delete rating
|
||||
|
||||
# Scan Settings (All Authenticated Users)
|
||||
GET /api/library/scan-settings # Get user's scan settings
|
||||
PUT /api/library/scan-settings # Update scan settings
|
||||
{
|
||||
"scan_frequency_minutes": 60,
|
||||
"auto_scan_enabled": true
|
||||
}
|
||||
|
||||
# Token Management
|
||||
POST /api/auth/refresh # Refresh access token
|
||||
POST /api/auth/logout # Logout (revokes refresh token)
|
||||
@@ -192,15 +212,48 @@ POST /api/auth/logout # Logout (revokes refresh token)
|
||||
## 📖 Media Support
|
||||
|
||||
### Ebooks
|
||||
- **Formats**: EPUB (full metadata), PDF, MOBI, AZW, TXT, DOC, FB2
|
||||
- **Formats**: EPUB (full metadata), PDF, MOBI, AZW, AZW3, TXT, DOC, DOCX, LIT, FB2, PDB, RTF
|
||||
- **Metadata**: Automatic extraction from EPUB files with Calibre support
|
||||
- **ISBN Support**:
|
||||
- Automatic normalization of ISBN-10 and ISBN-13 formats
|
||||
- Removes hyphens and spaces for consistent storage
|
||||
- Supports ISBN-10 with check digit X
|
||||
- Database function `normalize_isbn()` handles all conversions
|
||||
- **Reading**: Built-in web reader for EPUB files
|
||||
- **Library Requirement**: Ebooks require an existing ebook library; graceful error handling if none exists
|
||||
|
||||
### Comics
|
||||
- **Formats**: CBZ, CBR, CB7, CBT, PDF
|
||||
- **Structure**: Archive-based organization with chapter support
|
||||
- **Viewing**: Image extraction and web-based comic reader
|
||||
|
||||
## ⚙️ Background Processing
|
||||
|
||||
### Auto-Scanning
|
||||
- **Scheduled Scanning**: User-configurable scan frequency (per-user settings)
|
||||
- **Smart Scheduling**: Background scheduler manages scan jobs efficiently
|
||||
- **Per-User Settings**: Each user can enable/disable auto-scan and set frequency
|
||||
- **Resource Management**: Worker pool limits concurrent scanning operations
|
||||
|
||||
### Watch Mode
|
||||
- **Real-Time Monitoring**: File system watcher detects new files instantly
|
||||
- **Automatic Processing**: New media items processed and added to library
|
||||
- **Multi-Library Support**: Watch all library folders simultaneously
|
||||
- **Efficient**: Uses filesystem events for minimal resource usage
|
||||
|
||||
### Scan Settings API
|
||||
```bash
|
||||
# Get current scan settings
|
||||
GET /api/library/scan-settings
|
||||
|
||||
# Update scan settings
|
||||
PUT /api/library/scan-settings
|
||||
{
|
||||
"scan_frequency_minutes": 60, # How often to auto-scan (minutes)
|
||||
"auto_scan_enabled": true # Enable/disable auto-scanning
|
||||
}
|
||||
```
|
||||
|
||||
### Manga
|
||||
- **Formats**: CBZ, CBR (archives), PNG, JPG (image folders)
|
||||
- **Structure**: Archive support with folder-based image organization
|
||||
@@ -217,6 +270,10 @@ go run cmd/server/main.go
|
||||
# Frontend development
|
||||
npm run dev
|
||||
# Templates automatically recompile on changes
|
||||
|
||||
# Run tests
|
||||
go test ./... -v
|
||||
go test ./cmd/server/tests -v # Integration tests only
|
||||
```
|
||||
|
||||
### Environment
|
||||
@@ -240,8 +297,17 @@ services:
|
||||
- DBHOST=db
|
||||
depends_on:
|
||||
- db
|
||||
volumes:
|
||||
- /path/to/media:/media # Mount media directories
|
||||
- /path/to/uploads:/uploads # Upload directory
|
||||
```
|
||||
|
||||
### Auto-Starting Services
|
||||
The application automatically starts background services on startup:
|
||||
- **Scheduler**: Manages scheduled scanning jobs based on user settings
|
||||
- **Watch Mode**: Monitors all library folders for file system changes
|
||||
- **Worker Pool**: Processes scan jobs with configurable concurrency
|
||||
|
||||
### Database Schema
|
||||
- **Multi-Library Architecture**: Libraries, library types, folders, visibility tables
|
||||
- **Annotations System**: Notes, highlights with position tracking and color customization
|
||||
@@ -258,10 +324,13 @@ Complete API testing collection in `bruno/` directory:
|
||||
bruno/
|
||||
├── user/ # Authentication & profile endpoints
|
||||
├── admin/ # Admin-only operations
|
||||
├── library/ # Library management
|
||||
├── library/ # Library management
|
||||
├── media-items/ # Media content browsing
|
||||
├── ebooks/ # Ebook-specific operations
|
||||
├── notes/ # Notes API testing
|
||||
├── highlights/ # Highlights API testing
|
||||
├── scanner/ # Background scanning & watch mode
|
||||
├── progress/ # Reading progress tracking
|
||||
└── collection.bru # Main dashboard
|
||||
```
|
||||
|
||||
@@ -322,6 +391,8 @@ HTTP Status Codes:
|
||||
- **Row-Level Security**: User data isolation
|
||||
- **Secure Refresh Token Storage**: Encrypted token storage in database
|
||||
- **Automatic Token Cleanup**: Expired tokens cleaned up periodically
|
||||
- **ISBN Normalization**: Database function ensures consistent ISBN format
|
||||
- **Data Validation**: Server-side validation for all inputs including ISBNs
|
||||
|
||||
## 🔧 Configuration
|
||||
|
||||
@@ -350,6 +421,22 @@ INSERT INTO library_types (name, description, allowed_extensions) VALUES
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
### Integration Tests
|
||||
Comprehensive integration test suite in `cmd/server/tests/`:
|
||||
- Authentication & authorization tests
|
||||
- Library management tests
|
||||
- Media items operations
|
||||
- Notes & highlights functionality
|
||||
- ISBN normalization tests
|
||||
- Background scanning & watch mode tests
|
||||
- Edge cases and error handling
|
||||
- Security & rate limiting tests
|
||||
|
||||
Run integration tests:
|
||||
```bash
|
||||
go test ./cmd/server/tests -v
|
||||
```
|
||||
|
||||
### API Testing
|
||||
```bash
|
||||
# Install Bruno
|
||||
@@ -377,9 +464,20 @@ cmd/server/main.go # Application entry point
|
||||
│ ├── handlers/ # HTTP handlers
|
||||
│ │ ├── auth.go # Authentication & users
|
||||
│ │ ├── library.go # Library management
|
||||
│ │ └── ebook.go # Media operations
|
||||
│ └── services/ # Business logic
|
||||
│ └── library_service.go # Library service
|
||||
│ │ ├── ebook.go # Media operations
|
||||
│ │ └── refresh_token.go # Token management
|
||||
│ ├── services/ # Business logic
|
||||
│ │ ├── library_service.go # Library service
|
||||
│ │ ├── ebook_scanner.go # Media scanning
|
||||
│ │ ├── worker.go # Background job processing
|
||||
│ │ └── scheduler.go # Auto-scan scheduling
|
||||
│ └── middleware/ # HTTP middleware
|
||||
│ ├── rate_limiter.go # Rate limiting
|
||||
│ ├── login_attempts.go # Account lockout
|
||||
│ ├── password_validator.go # Password complexity
|
||||
│ ├── error_handler.go # Error handling
|
||||
│ ├── transaction.go # DB transactions
|
||||
│ └── request_tracing.go # Request logging
|
||||
templates/ # HTML templates with HTMX
|
||||
```
|
||||
|
||||
@@ -393,8 +491,12 @@ media_items # All media content (replaces ebooks table)
|
||||
media_ratings # User ratings for media items
|
||||
media_notes # User notes on media items
|
||||
media_highlights # User highlights with optional note links
|
||||
reading_progress # User reading progress
|
||||
users # User accounts and profiles
|
||||
reading_progress # User reading progress
|
||||
users # User accounts and profiles (includes scan settings)
|
||||
refresh_tokens # JWT refresh token storage
|
||||
|
||||
# Database Functions
|
||||
normalize_isbn() # ISBN normalization function
|
||||
|
||||
# Backward Compatibility Views
|
||||
ebook_notes # Notes view for ebook API compatibility
|
||||
@@ -403,6 +505,14 @@ ebook_highlights # Highlights view for ebook API compatibility
|
||||
|
||||
## 🎯 Future Roadmap
|
||||
|
||||
### Recently Added Features (v1.0)
|
||||
- ✅ **ISBN Normalization**: Automatic ISBN format standardization
|
||||
- ✅ **Background Scanning**: Scheduled library scanning with per-user settings
|
||||
- ✅ **Watch Mode**: Real-time file system monitoring for instant updates
|
||||
- ✅ **Integration Tests**: Comprehensive test suite for all major features
|
||||
- ✅ **Request Tracing**: Enhanced logging and debugging capabilities
|
||||
- ✅ **Graceful Library Handling**: Better error messages when libraries don't exist
|
||||
|
||||
### Media Type Extensions
|
||||
- **Audiobooks**: Audio file support with chapter tracking
|
||||
- **Podcasts**: RSS feed integration and automatic downloading
|
||||
@@ -424,6 +534,8 @@ ebook_highlights # Highlights view for ebook API compatibility
|
||||
- Implement proper error handling with pgx.ErrNoRows
|
||||
- Write comprehensive tests for new features
|
||||
- Update documentation for API changes
|
||||
- Add Bruno tests for new API endpoints
|
||||
- Include integration tests for complex features
|
||||
|
||||
### Code Style
|
||||
- Follow existing code formatting
|
||||
|
||||
Reference in New Issue
Block a user