📚 Bookmann

A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and Tailwind CSS featuring multiple library support, beautiful dark themes, and comprehensive media management.

✨ Features

🏗 Multi-Library System

• Multiple Media Types: Support for Ebooks, Comics, and Manga with modular architecture
• Per-Library Folders: Each library can have multiple scanning folders for flexible organization
• Library Visibility Control: Admins can control which libraries are visible to each user
• Type-Specific File Extensions:
  • Ebooks: .epub, .pdf, .mobi, .azw, .azw3, .txt, .rtf, .doc, .docx, .lit, .fb2, .pdb
  • Comics: .cbz, .cbr, .cb7, .cbt, .pdf
  • Manga: .cbz, .cbr, .png, .jpg, .jpeg, .gif, .bmp, .webp
• Easy Extension: Designed to add new media types in the future

🔒 Authentication & Security

• Multi-User Support: Complete user registration and authentication system
• JWT-Based Sessions: Secure token-based authentication with 1-hour expiration and refresh token support
• Role-Based Access: Admin and user roles with granular permission control
• Password Security: bcrypt hashing with complex password requirements:
  • Minimum 8 characters
  • At least one uppercase letter (A-Z)
  • At least one lowercase letter (a-z)
  • At least one number (0-9)
  • At least one special character (!@#$%^&*()_+-=[]{}|;':",./<>?)
• Account Lockout: Automatic account lockout after 5 failed login attempts (15-minute lockout period)
• Rate Limiting: Built-in rate limiting on auth endpoints (10 requests/minute) to prevent brute force attacks
• Refresh Tokens: Secure refresh token mechanism (7-day expiration) for seamless token renewal
• Input Validation: Comprehensive validation including username whitespace checks, email format validation
• Pagination Protection: Maximum pagination limits (1000 items) to prevent DoS attacks
• Path Validation: Library folder paths are validated for existence and accessibility
• Case-Insensitive Roles: Role values automatically normalized to lowercase
• Database Transactions: Multi-step database operations use transaction support for data consistency
• Standardized Error Responses: Consistent error format across all API endpoints

🎨 Beautiful UI

• 11 Dark Themes: Tokyo Night, Dracula, Nord, Solarized Dark, Monokai, One Dark Pro, Material Dark, Catppuccin variants
• Theme Persistence: User theme preferences saved to database
• Mobile-First Design: Fully responsive interface for all devices
• HTMX Integration: Dynamic interactions without page reloads

🔄 Universal Cross-Platform Sync (NEW!)

• Real-Time Progress Sync: Reading progress syncs automatically across all your devices
• Multi-Device Support: KOReader, Kobo e-readers, web browsers, mobile apps
• Format-Aware Tracking: EPUB CFI, page numbers, percentages, character offsets
• Highlights & Notes: Annotations sync universally across all devices
• Conflict Resolution: Smart handling when same book read on multiple devices
• Offline Support: Queue changes when offline, sync when reconnected
• Secure Device Auth: Web-based device approval, no passwords on devices
• Checkpoint Mode: Batch sync for better battery life
• WebSocket Updates: Real-time broadcast of progress changes

📱 Media Management

• Universal Media Support: Single system for all media types with unified interface
• Rich Metadata: Automatic extraction of title, author, series, publisher, ISBN, tags
• ISBN Normalization: Automatic ISBN format normalization (removes hyphens and spaces) supporting ISBN-10 and ISBN-13 formats
• Advanced Search:
  • Partial matching search across title, author, series, tags, and contributors
  • Automatic fuzzy search fallback when no partial matches found (handles typos and misspellings)
  • Real-time search results with highlighted matches
  • Keyboard navigation (↑↓ arrows, Enter to select, Escape to close)
  • Respects library visibility settings
• Dynamic Sorting: Sort your media collection by multiple fields
  • Title (A-Z or Z-A)
  • Author (A-Z or Z-A)
  • Date added (newest or oldest first)
  • Date published (newest or oldest first)
  • Copyright year (newest or oldest first)
  • Series order (with series number)
  • Page count (shortest or longest first)
  • Genre (A-Z or Z-A)
• Advanced Filtering: Filter media items with multiple options
  • Filter by author (partial match)
  • Filter by series (partial match)
  • Filter by genre (exact match)
  • Filter by language (English, Spanish, French, German, Japanese, Chinese, Korean, Russian, Italian, Portuguese)
  • Filter by copyright year range
  • Filter by items with cover images only
  • Combine multiple filters with URL state management for shareable links
• Enhanced Metadata: New fields for better organization
  • Language support for multi-lingual collections
  • Edition information (2nd Edition, Revised, Collector's Edition, etc.)
  • Page count for better sorting and progress calculation
  • External service integration (Goodreads, OpenLibrary, Google Books IDs)
  • Copyright year (distinct from publication date)
  • Structured genre classification
  • Subject tags (array of subjects)
• Advanced Rating: 5-star system with half-star precision (1-10 scale)
• Reading Progress: User-specific progress tracking with current page and total pages
• Notes & Highlights: Personal annotations and text highlighting with color customization
• Highlight Notes: Link highlights to detailed notes for comprehensive annotations
• Library Statistics: Media count and usage statistics per library
• Background Scanning: Automatic background scanning with configurable frequency per user
• Watch Mode: Real-time file system monitoring for instant library updates

🚀 Quick Start

Prerequisites

• Podman and Podman Compose (recommended) or Docker
• PostgreSQL database (handled by Podman)

Environment Setup

1. Create .env file:

cp .env.example

2. Edit .env with your secure values:

JWT_SECRET="your-secure-jwt-secret-key-here"
DBPASS="your-secure-database-password-here"

Running the Application

# Using Podman (recommended)
podman-compose up --build

# Using Docker
docker-compose up --build

# Option 2: Direct environment variables
export JWT_SECRET="your-secure-jwt-secret-key-here"
export DBPASS="your-secure-database-password-here"
podman-compose up --build

Access the application at: http://localhost:8765

👤 User Management

Role System

• Admin Users: Can create/manage libraries, add/delete media items, manage users
• Regular Users: Can view permitted libraries, rate items, track reading progress

First Admin Setup

The first user who registers automatically becomes an admin. For additional admins:

-- Connect to database
docker exec -it bookmann_db psql -U postgres -d bookmann

-- Promote user to admin
UPDATE users SET role = 'admin' WHERE email = 'user@example.com';

🏛 Library Management

Creating Libraries

1. Admin Access: Only administrators can create libraries
2. Library Types: Choose from Ebooks, Comics, or Manga
3. Multi-Folder Support: Add multiple scanning folders per library
4. Folder Organization: Organize your media collection across multiple paths

Library Visibility Control

• Admin Dashboard: Complete interface for managing library access
• User-Specific Control: Admins can show/hide libraries per user
• Personal Preferences: Admins can also hide libraries from their own view
• Simple Toggles: Checkbox-based interface for easy management

API Endpoints

Notes & Highlights (All Authenticated Users)

# Media Notes
GET /api/media-items/{id}/notes              # Get user's notes for media
POST /api/media-items/{id}/notes             # Create new note
GET /api/media-items/{id}/notes/{noteId}   # Get specific note
PUT /api/media-items/{id}/notes/{noteId}    # Update note
DELETE /api/media-items/{id}/notes/{noteId} # Delete note

# Media Highlights  
GET /api/media-items/{id}/highlights                   # Get user's highlights for media
POST /api/media-items/{id}/highlights                  # Create new highlight
GET /api/media-items/{id}/highlights/{highlightId}      # Get specific highlight
PUT /api/media-items/{id}/highlights/{highlightId}       # Update highlight
DELETE /api/media-items/{id}/highlights/{highlightId}   # Delete highlight

# Ebook Compatibility (Backward Compatible)
GET /api/ebooks/{id}/notes              # Get user's notes for ebook
POST /api/ebooks/{id}/notes             # Create new ebook note
PUT /api/ebooks/{id}/notes/{noteId}    # Update ebook note
DELETE /api/ebooks/{id}/notes/{noteId} # Delete ebook note

GET /api/ebooks/{id}/highlights                   # Get user's highlights for ebook
POST /api/ebooks/{id}/highlights                  # Create new ebook highlight
PUT /api/ebooks/{id}/highlights/{highlightId}       # Update ebook highlight
DELETE /api/ebooks/{id}/highlights/{highlightId}   # Delete ebook highlight

Media Search (All Authenticated Users)

# Search media items across all visible libraries
GET /api/media-items/search?q={query}

# Query Parameters:
# q (required): Search query (minimum 2 characters)
# - Searches across: title, author, series, tags, contributors
# - Partial matching: Case-insensitive substring search
# - Fuzzy fallback: Automatic when no partial matches found
# - Results ranked by relevance

# Examples:
GET /api/media-items/search?q=harry%20potter    # Search by title
GET /api/media-items/search?q=rowling          # Search by author
GET /api/media-items/search?q=hary%20poter     # Fuzzy search (handles typos)

# Response:
# - 200: Success (returns array of media items)
# - 404: No results found
# - 400: Missing or invalid query parameter
# - 401: Unauthorized

Library Management (Admin Only)

# Create new library
POST /api/libraries
{
  "name": "My Comic Collection",
  "description": "Digital comics and graphic novels",
  "type": "comics"
}

# List all libraries  
GET /api/libraries

# Add folder to library
POST /api/libraries/{library_id}/folders
{
  "folder_path": "/path/to/comics"
}

# Set library visibility for user
POST /api/libraries/visibility
{
  "library_id": "library-uuid",
  "is_visible": true
}

# Scanner Management (Admin Only)
POST /api/scanner/scan                  # Manual scan of all libraries
POST /api/scanner/start                 # Start scheduled scanning
POST /api/scanner/stop                  # Stop scheduled scanning
GET /api/scanner/status/{jobId}        # Get scan job status
POST /api/scanner/watch/start          # Start watch mode (real-time monitoring)
POST /api/scanner/watch/stop           # Stop watch mode
GET /api/scanner/watch/status          # Get watch mode status

User Access

# Get user's visible libraries
GET /api/libraries/visible

# Browse media in library
GET /api/media-items?library_id={library_id}&limit=20&offset=0

# Media Progress & Ratings
GET /api/media-items/{id}/progress     # Get reading progress
PUT /api/media-items/{id}/progress     # Update reading progress
GET /api/media-items/{id}/rating       # Get user rating
POST /api/media-items/{id}/rating      # Create/update rating
PUT /api/media-items/{id}/rating       # Update rating
DELETE /api/media-items/{id}/rating   # Delete rating

# Scan Settings (All Authenticated Users)
GET /api/library/scan-settings         # Get user's scan settings
PUT /api/library/scan-settings         # Update scan settings
{
  "scan_frequency_minutes": 60,
  "auto_scan_enabled": true
}

# Token Management
POST /api/auth/refresh                 # Refresh access token
POST /api/auth/logout                  # Logout (revokes refresh token)

📖 Media Support

Ebooks

• Formats: EPUB (full metadata), PDF, MOBI, AZW, AZW3, TXT, DOC, DOCX, LIT, FB2, PDB, RTF
• Metadata: Automatic extraction from EPUB files with Calibre support
• ISBN Support:
  • Automatic normalization of ISBN-10 and ISBN-13 formats
  • Removes hyphens and spaces for consistent storage
  • Supports ISBN-10 with check digit X
  • Database function normalize_isbn() handles all conversions
• Reading: Built-in web reader for EPUB files
• Library Requirement: Ebooks require an existing ebook library; graceful error handling if none exists

Comics

• Formats: CBZ, CBR, CB7, CBT, PDF
• Structure: Archive-based organization with chapter support
• Viewing: Image extraction and web-based comic reader

⚙️ Background Processing

Auto-Scanning

• Scheduled Scanning: User-configurable scan frequency (per-user settings)
• Smart Scheduling: Background scheduler manages scan jobs efficiently
• Per-User Settings: Each user can enable/disable auto-scan and set frequency
• Resource Management: Worker pool limits concurrent scanning operations

Watch Mode

• Real-Time Monitoring: File system watcher detects new files instantly
• Automatic Processing: New media items processed and added to library
• Multi-Library Support: Watch all library folders simultaneously
• Efficient: Uses filesystem events for minimal resource usage

Scan Settings API

# Get current scan settings
GET /api/library/scan-settings

# Update scan settings
PUT /api/library/scan-settings
{
  "scan_frequency_minutes": 60,    # How often to auto-scan (minutes)
  "auto_scan_enabled": true        # Enable/disable auto-scanning
}

Manga

• Formats: CBZ, CBR (archives), PNG, JPG (image folders)
• Structure: Archive support with folder-based image organization
• Viewing: Chapter-by-chapter viewing with page navigation

🎨 Development

Local Development

# Backend development
go mod tidy
go run cmd/server/main.go

# Frontend development
npm run dev
# Templates automatically recompile on changes

# Run tests
go test ./... -v
go test ./cmd/server/tests -v  # Integration tests only

Environment

• Backend: Go 1.25+ with pgx v5 for database operations
• Frontend: Tailwind CSS with HTMX for dynamic interactions
• Database: PostgreSQL 15+ with pgx v5 driver
• Authentication: JWT tokens with bcrypt password hashing

🐳 Deployment

For detailed deployment instructions, Docker configuration, and troubleshooting common issues, see TROUBLESHOOTING.md.

The application automatically starts background services on startup:

• Scheduler: Manages scheduled scanning jobs based on user settings
• Watch Mode: Monitors all library folders for file system changes
• Worker Pool: Processes scan jobs with configurable concurrency

Database Schema

• Multi-Library Architecture: Libraries, library types, folders, visibility tables
• Annotations System: Notes, highlights with position tracking and color customization
• Backward Compatibility: Views maintain existing API contracts for ebooks
• Type Safety: pgx v5 with proper error handling
• Migration Ready: Schema designed for easy future extensions

🧪 API Documentation

Bruno Testing Collection

Complete API testing collection in bruno/ directory:

bruno/
├── user/              # Authentication & profile endpoints
├── admin/          # Admin-only operations  
├── library/        # Library management
├── media-items/     # Media content browsing
├── ebooks/         # Ebook-specific operations
├── notes/          # Notes API testing
├── highlights/     # Highlights API testing
├── scanner/        # Background scanning & watch mode
├── progress/       # Reading progress tracking
├── devices/        # Device management
├── conflicts/       # Sync conflict resolution
├── queue/          # Sync queue management
├── sync-koreader/ # KOReader sync protocol
├── sync-kobo/       # Kobo sync protocol
└── collection.bru   # Main dashboard file

Authentication Flow

1. Register: POST /api/auth/register → JWT token
2. Login: POST /api/auth/login → JWT token
3. Protected Routes: Use Authorization: Bearer {token} header
4. Refresh Token: POST /api/auth/refresh → JWT token
5. Logout: POST /api/auth/logout → Revoke refresh token

API Reference Documentation

See API_REFERENCE.md for complete API documentation including:

• All endpoints with request/response examples
• Bruno v3.0 test collections for all endpoints
• Error handling details
• Authentication & security features

Device Setup Guides

• KOBOREADER_SETUP.md - KOReader device configuration
• KOBO_SETUP.md - Kobo device configuration

Universal Sync Documentation

• UNIVERSAL_SYNC_IMPLEMENTATION_GUIDE.md - Complete sync architecture

Error Handling

All API endpoints return standardized error responses:

{
  "error": "Error message",
  "message": "Detailed error information (if available)",
  "code": "Error code (if applicable)"
}

HTTP Status Codes:

• 400: Bad request (validation errors)
• 401: Unauthorized (invalid/missing token)
• 403: Forbidden (insufficient permissions)
• 404: Resource not found
• 429: Too many requests (rate limit exceeded)
• 500: Internal server error

🛡 Security Features

Authentication & Authorization

• JWT Tokens:
  • Short-lived access tokens (1-hour expiration)
  • Refresh tokens (7-day expiration) for seamless session renewal
  • Secure token storage and transmission
• Password Security:
  • bcrypt hashing with cost factor 12
  • Complex password requirements enforced
  • Password validation on registration and updates
• Account Protection:
  • Automatic lockout after 5 failed login attempts
  • 15-minute lockout period with countdown
  • IP-based and username-based attempt tracking
• Rate Limiting: 10 requests per minute on authentication endpoints
• Input Validation: Comprehensive server-side validation for all inputs
• CSRF Protection: Built-in with HTMX
• Role-Based Access Control: Admin and user roles enforced on all endpoints

Authorization

• Role-Based Access: Admin vs user permissions
• Library Visibility: Per-user library access control
• Admin Middleware: Protected routes for admin operations
• Content Security: XSS protection and secure headers

Database Security

• Parameterized Queries: SQL injection prevention with pgx v5
• Connection Pooling: Efficient database connection management
• Transaction Support: Multi-step operations wrapped in transactions
• Environment Variables: Secure secret management
• Row-Level Security: User data isolation
• Secure Refresh Token Storage: Encrypted token storage in database
• Automatic Token Cleanup: Expired tokens cleaned up periodically
• ISBN Normalization: Database function ensures consistent ISBN format
• Data Validation: Server-side validation for all inputs including ISBNs

🔧 Configuration

Environment Variables

# Required
JWT_SECRET=           # JWT signing secret (64-byte random string)
DBHOST=db            # PostgreSQL database host

# Optional  
SERVER_PORT=8765      # Application port
NODE_ENV=development   # Environment mode

Database Configuration

-- Library types are automatically seeded
INSERT INTO library_types (name, description, allowed_extensions) VALUES
('ebooks', 'Ebook files including EPUB, PDF, MOBI, etc.', 
 ARRAY['.epub', '.pdf', '.mobi', '.azw', '.azw3', '.txt', '.rtf', '.doc', '.docx', '.lit', '.fb2', '.pdb']),
('comics', 'Comic book archives and image formats', 
 ARRAY['.cbz', '.cbr', '.cb7', '.cbt', '.pdf']),
('manga', 'Manga files including archives and image folders', 
 ARRAY['.cbz', '.cbr', '.png', '.jpg', '.jpeg', '.gif', '.bmp', '.webp']);

🧪 Testing

Integration Tests

Comprehensive integration test suite in cmd/server/tests/:

• Authentication & authorization tests
• Library management tests
• Media items operations
• Notes & highlights functionality
• ISBN normalization tests
• Background scanning & watch mode tests
• Edge cases and error handling
• Security & rate limiting tests

Run integration tests:

# Standard test run (may hit rate limits)
go test ./cmd/server/tests -v

# Recommended: Run with test mode enabled to avoid rate limiting
TEST_MODE=true RATE_LIMIT_ENABLED=false go test ./cmd/server/tests -run TestIntegrationAPI -v

# Or with increased rate limits
TEST_MODE=true REQUESTS_PER_MINUTE=1000 go test ./cmd/server/tests -run TestIntegrationAPI -v

Test Configuration Options

The following environment variables can be used to configure test behavior:

• TEST_MODE: Set to true to enable test mode (logs additional info)
• RATE_LIMIT_ENABLED: Set to false to disable rate limiting for tests
• REQUESTS_PER_MINUTE: Increase rate limit (e.g., 1000) to avoid throttling

⚠️ WARNING: Never disable rate limiting or enable test mode in production environments. These settings are only for integration testing.

API Testing

# Install Bruno
npm install -g @usebruno/cli

# Run tests
bruno run

Unit Tests

go test ./...

📊 Architecture

Backend

cmd/server/main.go              # Application entry point
├── internal/
│   ├── config/             # Configuration management
│   ├── database/           # Database operations (SQLC generated)
│   │   ├── queries/     # SQL queries
│   │   └── models.go    # Generated models
│   ├── handlers/           # HTTP handlers
│   │   ├── auth.go     # Authentication & users
│   │   ├── library.go  # Library management
│   │   ├── ebook.go    # Media operations
│   │   └── refresh_token.go # Token management
│   ├── services/          # Business logic
│   │   ├── library_service.go  # Library service
│   │   ├── ebook_scanner.go    # Media scanning
│   │   ├── worker.go           # Background job processing
│   │   └── scheduler.go        # Auto-scan scheduling
│   └── middleware/         # HTTP middleware
│       ├── rate_limiter.go     # Rate limiting
│       ├── login_attempts.go   # Account lockout
│       ├── password_validator.go # Password complexity
│       ├── error_handler.go    # Error handling
│       ├── transaction.go      # DB transactions
│       └── request_tracing.go  # Request logging
templates/                   # HTML templates with HTMX

Database

libraries              # Library definitions
library_types          # Media type definitions (ebooks, comics, manga)
library_folders        # Folders per library
library_visibility     # User library access control
media_items           # All media content (replaces ebooks table)
media_ratings         # User ratings for media items
media_notes           # User notes on media items
media_highlights      # User highlights with optional note links
reading_progress      # User reading progress
users                 # User accounts and profiles (includes scan settings)
refresh_tokens        # JWT refresh token storage

# Database Functions
normalize_isbn()      # ISBN normalization function

# Backward Compatibility Views
ebook_notes           # Notes view for ebook API compatibility
ebook_highlights      # Highlights view for ebook API compatibility

🎯 Future Roadmap

Recently Added Features (v1.0)

• ✅ ISBN Normalization: Automatic ISBN format standardization
• ✅ Background Scanning: Scheduled library scanning with per-user settings
• ✅ Watch Mode: Real-time file system monitoring for instant updates
• ✅ Integration Tests: Comprehensive test suite for all major features
• ✅ Request Tracing: Enhanced logging and debugging capabilities
• ✅ Graceful Library Handling: Better error messages when libraries don't exist

Media Type Extensions

• Audiobooks: Audio file support with chapter tracking
• Podcasts: RSS feed integration and automatic downloading
• Video: Movie/TV series management with metadata
• Music: Album and track management with artwork

Advanced Features

• Mobile App: React Native mobile application
• API v2: GraphQL API for efficient data fetching
• Webhooks: External service integrations
• Analytics: Usage statistics and reporting
• Backup/Restore: Library backup and migration tools

📝 Contributing

Development Guidelines

• Follow Go best practices and effective Go
• Use pgx v5 for all database operations
• Implement proper error handling with pgx.ErrNoRows
• Write comprehensive tests for new features
• Update documentation for API changes
• Add Bruno tests for new API endpoints
• Include integration tests for complex features

Code Style

• Follow existing code formatting
• Use meaningful variable and function names
• Add comments for complex business logic
• Ensure type safety with proper error handling

📄 License

GPL-3.0 - See LICENSE file for details.

---

Built with ❤️ using Go, PostgreSQL, HTMX, and Tailwind CSS