john-okeefe
7ff565a068
CRITICAL FIXES (would cause test failures):
1. Integration test timing: 5s → 12s
- Test waited 5s but implementation uses 10s batch delay
- Would fail intermittently detecting all 20 files
2. Race condition in fileStability map access
- Lock released between check and insert (lines 342-352)
- Concurrent calls could create duplicate map entries
- Fixed by holding lock during entire function
3. Missing concurrency protection in scanDirectory()
- Multiple scans of same directory could run simultaneously
- Could cause race conditions in fileStability map
- Fixed with activeScans map to prevent duplicate scans
MAJOR FIXES (production issues under load):
4. Unbounded goroutine spawn
- Spawns unlimited goroutines for directory scans
- 100 changed directories = 100 concurrent scans = 1000s of goroutines
- Fixed with semaphore limiting concurrent scans to 10
- You correctly identified this as the same problem Phase 1 job queue solved
5. Memory leak in fileStability map
- Entries never cleaned up if waitForFileStability() called concurrently
- Fixed by proper lock pattern and cleanup on all code paths
6. No initial scan of root folders
- Only watches for NEW changes, misses existing files
- Fixed by adding performInitialScan() function
MEDIUM FIXES (edge cases / code quality):
7. Removed unused batchTimeout variable
- Was declared but never actually used
8. Completed smart event merging
- Added sibling directory consolidation logic
- Prevents redundant scans of sibling folders
9. Clarified subdirectory handling
- Updated comment to explain subdirs trigger own events
- scanDirectory() doesn't walk into them (by design)
10. Added cleanup on shutdown
- New Close() method cleans up all maps
- Waits for active scans with 5-second timeout
11. Test timing: 11s → 15s
- Prevents flaky tests under load
12. Added database error handling
- Checks libraryID.Valid before scanning
- Handles orphaned folders gracefully
NEW CODE ADDED:
- scanSemaphore chan struct{} - limits concurrent scans to 10
- activeScans map[string]bool - prevents duplicate scans
- activeScansMu sync.Mutex - protects activeScans
- performInitialScan() - scans root folders on startup
- Close() method - cleanup and graceful shutdown
ARCHITECTURAL IMPROVEMENT:
- Semaphore pattern (from Phase 1 job queue) applied at directory level
- Higher concurrency limit (10 directory scans vs 3 library scans)
- Prevents resource exhaustion while maintaining parallelism
- All map entries properly cleaned up (no memory leaks)
- Graceful shutdown with timeout
Document size: 3,130 lines (increased from 2,974 lines)
Total changes: 191 insertions, 35 deletions
📚 Bookhoard
A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and Tailwind CSS featuring universal cross-device sync, beautiful dark themes, and comprehensive media management.
✨ Why Bookhoard?
🔄 Universal Sync: Your reading progress, highlights, and notes sync automatically across all your devices - KOReader, Kobo, web, and mobile.
📱 Multi-Library: Organize your ebooks, comics, and manga with per-library folders and smart collections.
🎨 Beautiful UI: 11 gorgeous dark themes with responsive design that works on any device.
🔒 Secure: JWT authentication, bcrypt password hashing, rate limiting, and no passwords on devices.
🚀 Quick Start
Prerequisites
- Podman (recommended) or Docker
- 5 minutes of your time
Installation
# 1. Clone the repository
git clone https://github.com/yourusername/bookhoard.git
cd bookhoard
# 2. Set up environment
cp .env.example .env
# Generate secure passwords (no special characters):
# JWT_SECRET: openssl rand -hex 32
# DBPASS: openssl rand -hex 16
# Edit .env with your generated values
# 3. Start the server
podman-compose up --build -d # or: docker-compose up --build -d
# 4. Open your browser
open http://localhost:8765
The first user to register automatically becomes an admin.
📖 Key Features
Universal Cross-Platform Sync
- Real-Time Progress: Turn a page on your Kindle, see it on your phone
- Format-Aware: EPUB CFI, page numbers, percentages - all handled correctly
- Offline Queue: Changes sync when you reconnect, priority-processed
- Conflict Resolution: Smart handling when same book read on multiple devices
- Book Matching: Automatic matching using SHA-256, ISBN, UUID
- OPDS Catalog: Wireless book delivery to e-readers over Wi-Fi
- Format Conversion: On-the-fly EPUB→KEPUB for Kobo devices
Media Management
- Smart Search: Partial matching with fuzzy search fallback for typos
- Advanced Filtering: Filter by author, series, genre, language, year, cover images
- Dynamic Sorting: By title, author, date added, published date, page count, series
- Rich Metadata: Title, author, series, publisher, ISBN, language, edition, tags
- 5-Star Ratings: Half-star precision (1-10 scale)
- Notes & Highlights: Color-coded annotations with linked notes
- Usage Analytics: Reading statistics, device usage, popular books
Smart Collections
- Auto-Assign Rules: Automatically add books based on genre, author, series, tags, language, publisher, year
- Device Shelf Mappings: Sync collections to Kobo shelves and KOReader categories
- Test Before Creating: Preview which books match your rules
Library Organization
- Multi-Library Support: Ebooks, Comics, and Manga with type-specific file formats
- Multiple Folders: Add multiple scanning folders per library
- Visibility Control: Admins control which libraries each user can see
- Background Scanning: Auto-scan with per-user frequency settings
- Watch Mode: Real-time file system monitoring for instant updates
Security
- JWT Authentication: Short-lived access tokens (1 hour) with refresh tokens (7 days)
- Strong Passwords: Complexity requirements enforced (8+ chars, uppercase, lowercase, number, special)
- Account Lockout: 5 failed attempts = 15-minute lockout
- Rate Limiting: 10 requests/minute on auth endpoints
- Input Validation: Comprehensive validation on all inputs
- No Passwords on Devices: Web-based device approval with QR codes
📚 Documentation
For Users & Self-Hosters
- docs/user/sync-guide.md - Understanding and using universal sync
- docs/user/devices/kobo-setup.md - Kobo e-reader configuration
- docs/user/devices/koreader-setup.md - KOReader configuration
- docs/user/user-guide.md - General user guide
- docs/user/admin-guide.md - Admin features and configuration
- docs/user/settings-guide.md - Settings and preferences
For Developers
- docs/developer/api/api-reference.md - Complete API documentation
- docs/contributing/DEVELOPMENT.md - Development workflow
🎯 Supported Devices
| Platform | Sync | OPDS | Status |
|---|---|---|---|
| Web Browser | ✅ | ✅ | Full support |
| KOReader | ✅ | ✅ | Kindle, Kobo, PocketBook |
| Kobo Devices | ✅ | ✅ | Clara, Libra, Sage, etc. |
| Mobile Apps | 🚧 | 🚧 | Coming Q2 2026 |
🛠 Tech Stack
- Backend: Go 1.25+ with Echo framework
- Database: PostgreSQL 15+ with pgx v5
- Frontend: HTMX + Tailwind CSS + Templ
- Auth: JWT tokens with bcrypt password hashing
- Container: Podman (Docker compatible)
🧪 Testing
# Run all tests
make test-all
# Run integration tests (with test mode)
make test-integration
# Run Bruno OpenCollection YAML API tests
npm install -g @usebruno/cli
bruno run
📊 Project Status
Version: 1.0
License: GPL-3.0
Status: Production-ready ✅
🤝 Contributing
We welcome contributions! Please see docs/DEVELOPMENT.md for guidelines.
📄 License
GPL-3.0 - See LICENSE file for details.
Built with ❤️ using Go, PostgreSQL, HTMX, and Tailwind CSS