docs: rename DEVELOPMENT.md to development.md and update links

- Rename docs/contributing/DEVELOPMENT.md to development.md (lowercase) - Update all references from DEVELOPMENT.md to Development.md (titlecase links) - Update docs/contributing/contributing.md - Update docs/index.md
2026-02-03 13:54:11 -05:00
parent 3cd378bbee
commit d9c6be1429
12 changed files with 1376 additions and 404 deletions
@@ -0,0 +1,445 @@
+# Bookhoard Development Guide
+
+This guide is for developers contributing to Bookhoard or setting up a development environment.
+
+## 🏗 Architecture
+
+### Directory Structure
+
+```
+bookhoard/
+├── cmd/server/                    # Application entry point
+│   ├── main.go                   # Server initialization, route registration
+│   └── tests/                    # Integration tests (30+ test files)
+├── internal/
+│   ├── config/                   # Configuration management
+│   ├── database/                 # Database layer (SQLC generated)
+│   ├── handlers/                 # HTTP request handlers (18 files)
+│   ├── middleware/              # HTTP middleware (9 files)
+│   ├── services/                # Business logic (7 files)
+│   ├── sync/                    # Sync framework (5 files)
+│   ├── opds/                    # OPDS feed generation
+│   └── utils/                   # Utility functions
+├── templates/                   # UI templates (17 .templ files)
+├── web/src/                     # Frontend TypeScript
+├── database/schema/             # Database schema
+├── docs/                        # Documentation
+└── bruno/                       # API test collections
+```
+
+### Backend Components
+
+**Handlers** (`internal/handlers/`):
+- `auth.go` - Authentication & user management
+- `library.go` - Library CRUD operations
+- `ebook.go` - Media item operations
+- `media.go` - Media downloads, shelves
+- `koreader.go` - KOReader sync protocol
+- `kobo.go` - Kobo sync protocol
+- `collections.go` - Collection management
+- `devices.go` - Device registration/management
+- `conflicts.go` - Sync conflict resolution
+- `queue.go` - Sync queue management
+- `progress.go` - Reading progress tracking
+- `analytics.go` - Usage analytics
+- `opds.go` - OPDS feed generation
+- `websocket.go` - WebSocket connections
+- `sync.go` - Sync orchestration
+- `book_matching.go` - Book linking/matching
+- `sidecar.go` - Sidecar file handling
+- `refresh_token.go` - Token refresh logic
+- `context.go` - Handler context utilities
+
+**Middleware** (`internal/middleware/`):
+- `device_auth.go` - Device authentication
+- `device_rate_limiter.go` - Device-specific rate limiting
+- `error_handler.go` - Global error handling
+- `login_attempts.go` - Login attempt tracking
+- `password_validator.go` - Password complexity validation
+- `rate_limiter.go` - IP-based rate limiting
+- `request_tracing.go` - Request ID tracking
+- `security.go` - Security headers
+- `transaction.go` - Database transaction middleware
+
+**Services** (`internal/services/`):
+- `library_service.go` - Library operations
+- `ebook_scanner.go` - File scanning & metadata extraction
+- `worker.go` - Job queue worker pool
+- `scheduler.go` - Scheduled task manager
+- `collection_service.go` - Collection rules processing
+- `conversion_service.go` - EPUB→KEPUB conversion
+- `book_matching.go` - Book matching algorithms
+
+**Sync Framework** (`internal/sync/`):
+- `queue.go` - Sync queue processor
+- `progress.go` - Universal progress format
+- `websocket.go` - Real-time sync broadcast
+- `offline.go` - Offline sync support
+- `format.go` - Format group conversion
+
+### Database Schema
+
+**Core Tables**:
+- `users` - User accounts with authentication and settings
+- `libraries` - Library definitions
+- `library_types` - Media type definitions (ebooks, comics, manga)
+- `library_folders` - Multiple folders per library
+- `library_visibility` - User-specific library access control
+- `media_items` - Universal media storage (replaces ebooks table)
+- `media_ratings` - User ratings (1-10 scale for half-star precision)
+- `media_notes` - User annotations
+- `media_highlights` - User highlights with color customization
+- `reading_progress` - Universal progress tracking across devices
+- `devices` - Device registry for sync
+- `sync_queue` - Offline sync support
+- `sync_conflicts` - Conflict resolution tracking
+- `collections` - Device-neutral collections
+- `collection_items` - Books in collections
+- `device_shelf_mappings` - Map collections to device-specific shelves
+- `device_catalogs` - Track OPDS downloads and ContentId mappings
+- `kobo_shelves` - Kobo-specific shelf management
+- `reading_history` - Reading session tracking
+- `unlinked_books` - Track books that couldn't be auto-matched
+- `media_item_formats` - Track all format versions with hashes
+- `device_file_aliases` - Track file paths per device
+- `refresh_tokens` - JWT refresh token storage
+
+**Database Functions**:
+- `normalize_isbn()` - ISBN format normalization
+- `detect_format_group()` - Detect format group (reflowable, fixed_layout, comic_archive)
+- `convert_progress()` - Convert progress between format groups
+- `detect_conflict()` - Detect sync conflicts
+- `merge_progress()` - Merge progress from multiple sources
+
+### Technology Stack
+
+**Backend**:
+- Go 1.25+
+- Echo v4 - HTTP framework
+- pgx v5 - PostgreSQL driver
+- SQLC - SQL code generation
+- jwt-go - JWT authentication
+- bcrypt - Password hashing
+
+**Frontend**:
+- Templ - HTML templating with Go
+- HTMX - Dynamic interactions
+- Tailwind CSS - Styling
+- TypeScript - Frontend logic
+
+**Database**:
+- PostgreSQL 15+
+- 30+ tables
+- 50+ indexes
+- JSONB for complex data
+
+**Testing**:
+- Testify - Testing framework
+- Bruno - API testing
+- 30+ integration test files
+
+## 🚀 Local Development
+
+### Prerequisites
+
+- Go 1.25+
+- Node.js 18+ (for frontend build)
+- Podman or Docker
+- PostgreSQL 15+ (or use Podman)
+
+### Setup
+
+```bash
+# 1. Clone repository
+git clone https://github.com/yourusername/bookhoard.git
+cd bookhoard
+
+# 2. Install Go dependencies
+go mod download
+
+# 3. Install build tools
+go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest
+go install github.com/a-h/templ/cmd/templ@latest
+
+# 4. Set up environment
+cp .env.example .env
+# Edit .env with your settings
+
+# 5. Generate database code
+cd internal/database
+sqlc generate
+
+# 6. Generate templates
+cd ../../templates
+templ generate
+
+# 7. Build frontend
+cd ../web
+npm install
+npm run build
+
+# 8. Run tests
+cd ..
+go test ./... -v
+```
+
+### Running Locally
+
+```bash
+# Option 1: Using containers (recommended)
+podman-compose up --build
+
+# Option 2: Direct Go run (requires local PostgreSQL)
+# Generate secure passwords (no special characters):
+#   JWT_SECRET: openssl rand -hex 32
+#   DBPASS: openssl rand -hex 16
+export JWT_SECRET="your-dev-secret"
+export DBPASS="your-db-password"
+go run cmd/server/main.go
+```
+
+### Development Workflow
+
+**Backend Development**:
+```bash
+# Watch mode for Go (requires air or similar)
+air
+
+# Or manual rebuild
+go build -o bookhoard cmd/server/main.go
+./bookhoard
+```
+
+**Frontend Development**:
+```bash
+cd web
+npm run dev  # Watch mode for TypeScript/CSS
+```
+
+**Database Changes**:
+1. Edit `database/schema/schema.sql`
+2. Edit `internal/database/queries/queries.sql`
+3. Run: `cd internal/database && sqlc generate`
+4. Restart server
+
+**Template Changes**:
+1. Edit `templates/*.templ`
+2. Run: `cd templates && templ generate`
+3. Restart server (templates auto-reload in dev mode)
+
+## 🧪 Testing
+
+### Unit Tests
+
+```bash
+# Run all unit tests
+go test ./... -v
+
+# Run specific package tests
+go test ./internal/handlers/... -v
+
+# Run with coverage
+go test ./... -coverprofile=coverage.out
+go tool cover -html=coverage.out
+```
+
+### Integration Tests
+
+```bash
+# Run integration tests (may hit rate limits)
+go test ./cmd/server/tests -v
+
+# Run with test mode (recommended)
+TEST_MODE=true RATE_LIMIT_ENABLED=false go test ./cmd/server/tests -v
+
+# Run specific test
+go test ./cmd/server/tests -run TestAuth -v
+```
+
+### API Testing with Bruno
+
+```bash
+# Install Bruno CLI
+npm install -g @usebruno/cli
+
+# Run all tests
+bruno run
+
+# Run specific collection
+bruno run bruno/user/
+bruno run bruno/sync-kobo/
+```
+
+### Test Configuration
+
+Environment variables for testing:
+- `TEST_MODE=true` - Enable test mode (disables rate limiting)
+- `RATE_LIMIT_ENABLED=false` - Disable rate limiting
+- `REQUESTS_PER_MINUTE=1000` - Increase rate limit
+
+**⚠️ WARNING**: Never enable these in production!
+
+## 📝 Code Style Guidelines
+
+### Go Code
+
+- Follow [Effective Go](https://go.dev/doc/effective_go) guidelines
+- Use `gofmt` for formatting
+- Use `golangci-lint` for linting
+- Use meaningful variable and function names
+- Add comments for complex business logic
+- Handle errors explicitly - don't ignore them
+
+### Database Operations
+
+- **Always use sqlc-generated code** - No raw SQL in handlers
+- Use transactions for multi-step operations
+- Handle `pgx.ErrNoRows` explicitly
+- Use `pgtype.UUID` for UUID parameters
+- Validate inputs before database operations
+
+### Error Handling
+
+```go
+// Good - Explicit error handling
+user, err := h.db.GetUser(c.Request().Context(), userID)
+if err != nil {
+    if errors.Is(err, pgx.ErrNoRows) {
+        return c.JSON(http.StatusNotFound, map[string]string{"error": "user not found"})
+    }
+    return c.JSON(http.StatusInternalServerError, map[string]string{"error": "database error"})
+}
+
+// Bad - Ignoring errors
+user, _ := h.db.GetUser(c.Request().Context(), userID)
+```
+
+### Adding New Features
+
+1. **Database First**: Add tables/columns to `schema.sql`
+2. **Generate Queries**: Add to `queries.sql` and run `sqlc generate`
+3. **Handler**: Implement in `internal/handlers/`
+4. **Routes**: Register in `cmd/server/main.go`
+5. **Tests**: Add integration test in `cmd/server/tests/`
+6. **Bruno**: Add API test in `bruno/`
+7. **Docs**: Update relevant documentation
+
+### API Design Principles
+
+- RESTful naming conventions
+- Consistent error responses
+- Proper HTTP status codes
+- JWT authentication on protected routes
+- Input validation with struct tags
+- Use echo.Context for request/response
+
+## 🐳 Deployment
+
+### Building for Production
+
+```bash
+# Using Makefile
+make build-force
+
+# Or manually
+podman-compose build --no-cache
+```
+
+### Environment Variables
+
+Required for production:
+- `JWT_SECRET` - 64-byte random string (generate: `openssl rand -hex 32`)
+- `DBPASS` - Strong database password (generate: `openssl rand -hex 16`)
+- `BASE_URL` - Public URL (e.g., https://bookhoard.example.com)
+
+Optional:
+- `HTTPS_PROXY` - If behind reverse proxy
+
+**Note**: Conversion service, rate limiting, and other operational settings have defaults in `docker-compose.yml` and can be overridden via `.env` if needed.
+
+### Performance Tuning
+
+**PostgreSQL Settings**:
+```sql
+-- In postgresql.conf
+shared_buffers = 256MB
+effective_cache_size = 1GB
+maintenance_work_mem = 64MB
+checkpoint_completion_target = 0.9
+wal_buffers = 16MB
+default_statistics_target = 100
+random_page_cost = 1.1
+effective_io_concurrency = 200
+work_mem = 2621kB
+min_wal_size = 1GB
+max_wal_size = 4GB
+```
+
+**Go Settings**:
+- GOMAXPROCS = number of CPU cores
+- Worker pool concurrency: 3 (configurable in services/worker.go)
+
+## 🔍 Debugging
+
+### Enable Debug Logging
+
+```bash
+# Set environment variable
+export DEBUG=true
+
+# Or in .env
+DEBUG=true
+```
+
+### Common Issues
+
+**Database Connection Errors**:
+- Check PostgreSQL is running
+- Verify DATABASE_HOST and DATABASE_PORT
+- Check firewall settings
+
+**Rate Limiting During Development**:
+- Enable test mode: `TEST_MODE=true RATE_LIMIT_ENABLED=false`
+- Or increase limit: `REQUESTS_PER_MINUTE=1000`
+
+**Template Not Updating**:
+- Run `templ generate` in templates/ directory
+- Restart server
+
+**Database Queries Not Working**:
+- Run `sqlc generate` in internal/database/
+- Check generated code in `queries.sql.go`
+- Verify SQL syntax in `queries.sql`
+
+## 📚 Additional Resources
+
+- [Project Guidelines](../../PROJECT_GUIDELINES.md) - Development rules and standards
+- [API Reference](../API_REFERENCE.md) - Complete API documentation
+- [Troubleshooting](../TROUBLESHOOTING.md) - Deployment issues
+- [Go Documentation](https://go.dev/doc/)
+- [Echo Framework](https://echo.labstack.com/docs)
+- [pgx Documentation](https://pgx.github.io/pgx/)
+
+## 🤝 Contributing
+
+1. Fork the repository
+2. Create a feature branch (`git checkout -b feature/amazing-feature`)
+3. Follow code style guidelines
+4. Add tests for new features
+5. Ensure all tests pass
+6. Commit with clear messages
+7. Push to branch (`git push origin feature/amazing-feature`)
+8. Open a Pull Request
+
+### Pull Request Checklist
+
+- [ ] Code follows style guidelines
+- [ ] Tests added/updated
+- [ ] Documentation updated
+- [ ] All tests passing
+- [ ] No new warnings
+- [ ] Commit messages are clear
+
+---
+
+**Happy Coding!** 🚀