john-okeefe
59d5de3607
The bookhoard container crashed with 'panic: Failed to create file
watcher: too many open files' (media_scanner.go) after running for a few
hours, preceded by floods of 'no space left on device' from watcher.Add.
Root cause: every scan job called NewMediaScanner(), which eagerly
created an fsnotify watcher. SetFolders() then walked the entire library
tree and registered one inotify watch per directory (~3,000+ across the
libraries), and ScanFolders() registered them again during its walk. The
worker never called scanner.Close() on these ephemeral per-job scanners,
and the worker loop had no recover(), so:
1. Leaked watchers accumulated until the kernel inotify watch cap was
hit (ENOSPC -> 'no space left on device'), then
2. the process fd limit (ulimit -n 1024) was exhausted, causing
fsnotify.NewWatcher() to fail with EMFILE, and
3. NewMediaScanner panicked on that error, taking down the whole
process (exit code 2). With no restart policy the container stayed
down.
The scan jobs run frequently (scan_poll_interval), so the leak built up
within hours. Note this was NOT a disk-space issue; df showed plenty free.
Fix:
- media_scanner.go: NewMediaScanner no longer creates a watcher eagerly
(s.watcher starts nil), which removes the panic site entirely -- there
is nothing to fail at construction. The watcher is created lazily only
when needed.
- media_scanner.go: SetFolders gains a [?1049h[22;0;0t[1;24r(B[m[4l[?7h[?25l[H[2JEvery 2.0s: bool[1;37Hgaruda-ser8: Fri 31 Jul 2026 10:45:41 AM EDT[2;66Hin 0.002s (127)[2;80H
[3dsh: line 1: bool: command not found
[4d[24;1H[?12l[?25h[?1049l[23;0;0t
[?1l> parameter. It creates
and populates a watcher (returning an error instead of panicking) only
when watch=true; otherwise it skips all watcher.Add calls. ScanFolders
guards its watcher.Add with a nil check, and the WatchChanges event
loop exits cleanly when there is no watcher (polling still runs).
- worker.go: the worker() loop now wraps each job in defer/recover() so a
panicking job is recorded as failed and can never kill the process.
- worker.go: the three ephemeral scan handlers (processScanJob,
processSetFoldersJob, processDirectoryScanJob) now defer scanner.Close()
and call SetFolders(..., false), so scan jobs allocate zero watchers and
zero inotify watches. Any pre-existing leak is also bounded by Close().
- handlers/scanner.go: the long-lived watch-mode scanners (StartScanner
and StartWatchModeForLibrary) pass watch=true since they actually read
watcher.Events for live change detection.
- calibre_integration_test.go: updated to the new SetFolders signature
(watch=false, matching one-off scan usage).
Auto-add is fully preserved: new files are still detected by the periodic
poller (startBackupScan), which is independent of fsnotify and unaffected
by these changes. The watch-mode event loop remains as bonus responsiveness
when inotify is available; through Docker bind mounts where inotify is
unreliable, polling is what catches new books.
📚 Bookhoard
A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and Tailwind CSS featuring universal cross-device sync, beautiful dark themes, and comprehensive media management.
✨ Why Bookhoard?
🔄 Universal Sync: Your reading progress, highlights, and notes sync automatically across all your devices - KOReader, Kobo, web, and mobile.
📱 Multi-Library: Organize your ebooks, comics, and manga with per-library folders and smart collections.
🎨 Beautiful UI: 11 gorgeous dark themes with responsive design that works on any device.
🔒 Secure: JWT authentication, bcrypt password hashing, rate limiting, and no passwords on devices.
🚀 Quick Start
Prerequisites
- Podman (recommended) or Docker
- 5 minutes of your time
Installation
# 1. Clone the repository
git clone https://git.linuxhg.com/Bookhoard/bookhoard.git
cd bookhoard
# 2. Set up environment
cp .env.example .env
# Generate secure passwords (no special characters):
# JWT_SECRET: openssl rand -hex 32
# DBPASS: openssl rand -hex 16
# Edit .env with your generated values
# 3. Pull images and start the server
docker compose pull
docker compose up -d
# Optionally pin a specific version: set IMAGE_TAG in .env (defaults to "latest")
# 4. Open your browser
open http://localhost:8765
The first user to register automatically becomes an admin.
📖 Key Features
Universal Cross-Platform Sync
- Real-Time Progress: Turn a page on your Kindle, see it on your phone
- Format-Aware: EPUB CFI, page numbers, percentages - all handled correctly
- Offline Queue: Changes sync when you reconnect, priority-processed
- Conflict Resolution: Smart handling when same book read on multiple devices
- Book Matching: Automatic matching using SHA-256, ISBN, UUID
- OPDS Catalog: Wireless book delivery to e-readers over Wi-Fi
- Format Conversion: On-the-fly EPUB→KEPUB for Kobo devices
Media Management
- Calibre Integration: Automatic metadata import from Calibre
metadata.opfsidecar files - Smart Search: Partial matching with fuzzy search fallback for typos
- Advanced Filtering: Filter by author, series, genre, language, year, cover images
- Dynamic Sorting: By title, author, date added, published date, page count, series
- Rich Metadata: Title, author, series, publisher, ISBN, language, edition, tags
- 5-Star Ratings: Half-star precision (1-10 scale)
- Notes & Highlights: Color-coded annotations with linked notes
- Usage Analytics: Reading statistics, device usage, popular books
Smart Collections
- Auto-Assign Rules: Automatically add books based on genre, author, series, tags, language, publisher, year
- Device Shelf Mappings: Sync collections to Kobo shelves and KOReader categories
- Test Before Creating: Preview which books match your rules
Library Organization
- Multi-Library Support: Ebooks, Comics, and Manga with type-specific file formats
- Multiple Folders: Add multiple scanning folders per library
- Visibility Control: Admins control which libraries each user can see
- Background Scanning: Auto-scan with per-user frequency settings
- Watch Mode: Real-time file system monitoring for instant updates
Security
- JWT Authentication: Short-lived access tokens (1 hour) with refresh tokens (7 days)
- Strong Passwords: Complexity requirements enforced (8+ chars, uppercase, lowercase, number, special)
- Account Lockout: 5 failed attempts = 15-minute lockout
- Rate Limiting: 10 requests/minute on auth endpoints
- Input Validation: Comprehensive validation on all inputs
- No Passwords on Devices: Web-based device approval with QR codes
📚 Documentation
For Users & Self-Hosters
- docs/user/calibre-integration.md - Calibre library integration
- docs/user/sync-guide.md - Understanding and using universal sync
- docs/user/devices/kobo-setup.md - Kobo e-reader configuration
- docs/user/devices/koreader-setup.md - KOReader configuration
- docs/user/user-guide.md - General user guide
- docs/user/admin-guide.md - Admin features and configuration
- docs/user/settings-guide.md - Settings and preferences
For Developers
- docs/developer/api/api-reference.md - Complete API documentation
- docs/contributing/DEVELOPMENT.md - Development workflow
🎯 Supported Devices
| Platform | Sync | OPDS | Status |
|---|---|---|---|
| Web Browser | ✅ | ✅ | Full support |
| KOReader | ✅ | ✅ | Kindle, Kobo, PocketBook |
| Kobo Devices | ✅ | ✅ | Clara, Libra, Sage, etc. |
| Mobile Apps | 🚧 | 🚧 | Coming Q2 2026 |
🛠 Tech Stack
- Backend: Go 1.25+ with Echo framework
- Database: PostgreSQL 15+ with pgx v5
- Frontend: HTMX + Tailwind CSS + Templ
- Auth: JWT tokens with bcrypt password hashing
- Container: Podman (Docker compatible)
🧪 Testing
# Run all tests
make test-all
# Run integration tests (with test mode)
make test-integration
# Run Bruno OpenCollection YAML API tests
npm install -g @usebruno/cli
bruno run
📊 Project Status
Version: 1.0
License: GPL-3.0
Status: Production-ready ✅
🤝 Contributing
We welcome contributions! Please see docs/DEVELOPMENT.md for guidelines.
📄 License
GPL-3.0 - See LICENSE file for details.
Built with ❤️ using Go, PostgreSQL, HTMX, and Tailwind CSS