refactor: reorganize project structure and update configurations

- Move migrations/ to database/schema/ for clarity on database schema definitions - Move sqlc.yaml to internal/database/ to group with database code - Move static/ to cmd/server/static/ to co-locate with server - Update all configuration files and documentation - Follow Go project conventions for better organization
2026-01-24 23:40:31 -05:00
parent 8a951fb242
commit 08e80ae84b
34 changed files with 532 additions and 104 deletions
@@ -11,11 +11,15 @@ A self-hosted ebook management system built with Go, PostgreSQL, HTMX, and Tailw
 - **🌙 Multiple Themes**: 11 beautiful themes including Tokyo Night, Dracula, Nord, Solarized Dark, Monokai, One Dark Pro, Material Dark, and Catppuccin variants (Mocha, Macchiato, Frappé, Latte) with user preferences saved to database
 - **🎨 Theme Persistence**: User theme choices sync between browser and server
 - **📖 Reading Progress**: User-specific reading progress tracking
+- **⭐ User Ratings**: Rate and review ebooks with personalized rating system
 - **📁 Multiple Folder Support**: Configure multiple ebook folders per user for comprehensive library management
- **🔍 Smart Scanner**: Automatic ebook discovery with EPUB metadata extraction (title, author, description, publisher, etc.)
+- **🔍 Enhanced Scanner**: Intelligent ebook discovery with Calibre folder structure support and comprehensive metadata extraction
 - **👀 Real-Time Monitoring**: File system monitoring for automatic ebook detection and updates
- **📚 Rich Metadata**: Automatic extraction of ebook metadata from EPUB files
- **🔧 RESTful API**: Clean API endpoints with JWT authentication
+- **📚 Rich Metadata**: Automatic extraction of ebook metadata (title, author, description, publisher, series, ISBN, tags) from EPUB files with Calibre-specific support
+- **🏛️ Calibre Integration**: Full support for Calibre folder structures and metadata (calibre:series, calibre:series_index)
+- **📂 Smart Folder Detection**: Automatically detects Author/Book, Author/Series/Book, and Calibre naming conventions
+- **🔄 Subfolder Scanning**: Recursively scans subdirectories with proper folder structure analysis
+- **🔧 RESTful API**: Clean API endpoints with JWT authentication and proper error handling
 - **🐳 Docker Ready**: Single-container deployment with PostgreSQL
 - **🧪 API Testing**: Complete Bruno collection for testing all endpoints
 - **📱 Responsive Design**: Mobile-first responsive interface using Tailwind CSS
@@ -110,19 +114,82 @@ The application uses Go HTML templates for server-side rendering with HTMX for d
 - `GET /api/ebooks/:id/ratings` - Get all ratings for ebook

 ### Scanner (Protected)
- `POST /api/scanner/scan` - Manually scan configured folders for ebooks
+- `POST /api/scanner/scan` - Scan user's configured ebook folders (supports optional folder_paths parameter for testing)
 - `POST /api/scanner/start` - Start real-time monitoring of configured folders
 - `POST /api/scanner/stop` - Stop real-time folder monitoring

-## 📁 Ebook Scanner
+## 📁 Enhanced Ebook Scanner

-Bookmann includes an intelligent ebook scanner that can automatically discover and catalog ebooks from your configured folders.
+Bookmann includes an intelligent ebook scanner with full Calibre integration and smart folder structure detection.

 ### Setting Up Folders

-1. **Add Folders**: Use the API or Bruno to add ebook folders to your account
-2. **Supported Formats**: EPUB, PDF, MOBI, AZW3, FB2, TXT
-3. **Metadata Extraction**: EPUB files automatically get rich metadata (title, author, description, publisher, etc.)
+1. **Add Folders**: Use `POST /api/auth/ebook-folders` to add ebook folders to your account
+2. **Supported Formats**: EPUB (full metadata), PDF (basic), MOBI, AZW3, FB2, TXT
+3. **Calibre Integration**: Automatically recognizes Calibre folder structures and metadata
+
+### Folder Structure Support
+
+**Calibre Structure (Preferred)**
+- `Author Name/Book Title/` - Simple Calibre structure
+- `Author Name/Series Name/Book Title/` - Series-based structure  
+- `Author Name/Series Name, Book #1 - Book Title/` - Full Calibre naming with series numbers
+
+**Alternative Structures**
+- Flat folder structures (all ebooks in root folder)
+- Custom subfolder organization
+- Mixed structures (Calibre + custom folders)
+
+### Example Folder Structures
+
+**Calibre Standard**
+```
+Books/
+├── Brandon Sanderson/
+│   ├── Mistborn/
+│   │   ├── The Final Empire.epub
+│   │   └── The Well of Ascension.epub
+│   └── The Stormlight Archive/
+│       ├── The Way of Kings.epub
+│       └── Words of Radiance.epub
+└── Patrick Rothfuss/
+    └── The Kingkiller Chronicle/
+        ├── The Name of the Wind.epub
+        └── The Wise Man's Fear.epub
+```
+
+**Calibre with Series Numbers**
+```
+Books/
+├── Brandon Sanderson/
+│   ├── Mistborn Trilogy, Book #1 - The Final Empire/
+│   │   └── The Final Empire.epub
+│   └── Mistborn Trilogy, Book #2 - The Well of Ascension/
+│       └── The Well of Ascension.epub
+```
+
+### Metadata Extraction
+
+**File-based Metadata**
+- **EPUB**: Title, Author, Description, Publisher, Series, Series Number, ISBN, Tags, Contributors, Publish Date
+- **PDF**: Basic filename extraction (can be enhanced with PDF library)
+- **Other formats**: Filename as title
+
+**Folder-based Metadata (Fallback)**
+- Extracts author from folder name
+- Extracts series information from folder structure
+- Detects series numbers from folder names
+- Handles underscore-to-space conversion
+
+**Enhanced Features**
+- **Priority**: File metadata > Folder structure metadata > Filename fallback
+- **Subfolder watching**: Automatically watches new subdirectories
+- **Real-time updates**: Processes new/modified files immediately
+- **Calibre-specific support**: Reads `calibre:series` and `calibre:series_index` metadata
+
+**PDF & Other Formats**
+- Basic filename extraction
+- Folder structure metadata fallback

 ### Scanner Operations

@@ -130,12 +197,14 @@ Bookmann includes an intelligent ebook scanner that can automatically discover a
 - **Start Monitoring**: `POST /api/scanner/start` - Begin real-time monitoring for changes
 - **Stop Monitoring**: `POST /api/scanner/stop` - Stop monitoring (folders remain configured)

-### Real-Time Features
+### Advanced Features

- **Auto-Discovery**: New ebooks added to folders are automatically detected
- **Metadata Updates**: Modified files get updated metadata
- **Duplicate Prevention**: Existing ebooks are updated, not duplicated
- **Multi-Folder Support**: Monitor multiple directories simultaneously
+- **Subfolder Scanning**: Recursively scans all subdirectories
+- **Smart Error Handling**: Properly handles file system errors and database issues
+- **Metadata Priority**: File metadata → Folder structure → Filename fallback
+- **Real-Time Detection**: Automatic discovery of new and modified ebooks
+- **Duplicate Prevention**: Updates existing entries instead of creating duplicates
+- **Dynamic Watching**: Automatically watches new subdirectories as they're created

 ## API Testing

@@ -151,34 +220,45 @@ Use the included Bruno collection in the `bruno/` directory for testing the API:
 ```
 .
 ├── cmd/server/              # Application entry point
+│   ├── main.go             # Main server application
+│   └── static/             # Static web assets (CSS, JS, images)
 ├── internal/
 │   ├── config/              # Configuration management
 │   ├── database/            # Database connection and queries
 │   ├── handlers/            # HTTP handlers (auth + ebooks)
 │   └── services/            # Business logic services (ebook scanner)
-├── migrations/              # Database migrations
+├── database/schema/          # Database schema definitions
 ├── templates/               # Go HTML templates with HTMX
 ├── bruno/                   # Bruno API testing collection
 ├── Dockerfile               # Docker build configuration
 ├── docker-compose.yml       # Docker Compose setup
 ├── go.mod                   # Go module definition
 ├── go.sum                   # Go module checksums
-├── sqlc.yaml                # SQL code generation config
+├── internal/database/sqlc.yaml  # SQL code generation config
 └── README.md
 ```

 ## 🎨 Recent Enhancements

+### Major Scanner Improvements
+- **Calibre Integration**: Full support for Calibre folder structures and metadata fields
+- **Smart Folder Detection**: Automatically recognizes Author/Book, Author/Series/Book patterns
+- **Enhanced Metadata Extraction**: EPUB parsing with Calibre-specific support (calibre:series, calibre:series_index, ISBN, tags)
+- **Subfolder Scanning**: Recursive directory scanning with automatic new folder watching
+- **Robust Error Handling**: Proper pgx.ErrNoRows handling and comprehensive error recovery
+- **Folder-based Metadata**: Fallback metadata extraction from folder structures when file metadata is incomplete
+- **Multi-format Support**: EPUB (full), PDF (basic), MOBI, AZW3, FB2, TXT file formats
+
 ### Backend Improvements
 - **Server-Side Rendering**: Replaced static frontend with Go HTML templates
 - **Theme System**: Database-backed user theme preferences with 11 beautiful dark themes
 - **HTMX Integration**: Dynamic interactions using HTMX for modern UX
- **Enhanced Security**: JWT authentication with theme persistence
+- **Enhanced Security**: JWT authentication with proper error handling
 - **User Profile Management**: Full CRUD operations for user profiles, usernames, emails, passwords, and account deletion
- **Multiple Folder Support**: Users can configure multiple ebook directories
- **Smart Ebook Scanner**: Automatic discovery and metadata extraction from EPUB files
- **Real-Time Monitoring**: File system watching for automatic ebook updates
+- **Multiple Folder Support**: Users can configure multiple ebook directories with per-user folder management
+- **Real-Time Monitoring**: File system watching for automatic ebook updates with dynamic subfolder detection
 - **Scan Settings**: User-configurable scan frequency and auto-scan options
+- **Rating System**: User-specific ebook ratings with full CRUD operations

 ### Frontend Redesign
 - **Beautiful Homepage**: Hero section with features showcase and modern design
@@ -192,10 +272,11 @@ Use the included Bruno collection in the `bruno/` directory for testing the API:
 - **Go Templates**: Server-side rendering with template inheritance
 - **Tailwind CSS**: Utility-first CSS framework via CDN
 - **TypeScript Support**: Client-side scripting with TypeScript compilation
- **Database Schema**: Added user names (first_name, last_name) and user_ebook_folders table for enhanced user profiles and multiple folder support
- **API Expansion**: New endpoints for user profile management, folder management, scanner operations, and scan settings
- **Metadata Extraction**: EPUB parsing for rich ebook information
- **File System Monitoring**: Real-time folder watching with fsnotify
+- **Database Schema**: Enhanced with user profiles, user_ebook_folders, ebook_ratings tables
+- **API Expansion**: Comprehensive endpoints for user management, folder operations, scanner controls, and ratings
+- **Advanced Metadata**: Rich ebook information extraction with fallback strategies
+- **File System Monitoring**: Real-time folder watching with automatic new directory detection
+- **Error Recovery**: Robust database error handling with proper pgx integration

 ## License