bookhoard/PROJECT_GUIDELINES.md

# Combined Project Guidelines for Bookhoard

## 🚨 CRITICAL PROHIBITIONS (Never violate these)

### Backend & Database
- ❌ **NEVER modify backend code when working on frontend-only tasks**
- ❌ **NEVER modify database schema** unless explicitly instructed for full-stack changes
- ❌ **NEVER use Docker** - use Podman only
- ❌ **NEVER build server binaries locally** - all builds through Dockerfile/docker-compose
- ❌ **NEVER create new migration files** - merge changes into current one until release
- ❌ **NEVER use `git checkout` on schema files** without checking what will be lost
 - ❌ **NEVER break existing functionality** unless explicitly instructed
 - ❌ **NEVER duplicate business logic** - keep logic in services, not handlers
 - ❌ **NEVER bypass service layer** - all database operations must go through services
 
### Testing
- ✅ **ALWAYS use `setupTestServer()` helper from `cmd/server/tests/test_helpers.go`**
- ✅ **Share one test setup across all subtests** - call `setupTestServer()` once at test function level, not per subtest
- ✅ **Prefer table-driven tests** - use `t.Run()` with test cases instead of duplicate test functions
- ✅ **Configure database pools efficiently** - use `max_conns=1` for test pools (via `pgxpool.ParseConfig()`) to prevent connection exhaustion
- ❌ **NEVER create separate `pgxpool` per test** - each pool creates 4 connections by default; 78 tests = 312 potential connections > PostgreSQL's 100 limit
- ❌ **NEVER call `setupTestServer()` in loops or within subtests** - creates unnecessary database pools and exhausts connections
- ✅ **DO verify tests pass** - run full test suite before completing work
- ✅ **Use `t.Cleanup()` properly** - the `TestServerSetup` pattern automatically handles cleanup via `t.Cleanup()`

### Frontend & Styling
- ❌ **NEVER modify backend/API for frontend features without user confirmation**
- ❌ **NEVER use custom CSS** - TailwindCSS classes only
  - **⚠️ EXCEPTION**: `templates/error.templ` may have inline CSS because error pages must work when main app fails (404, server errors, CSS fails to load)
- ❌ **NEVER use JavaScript** - convert all to TypeScript
  - **⚠️ EXCEPTION**: Inline JS function calls in HTML attributes (e.g., `onclick="myFunction()"`) are acceptable for simple interactions
- ❌ **NEVER use Object-Oriented Programming** (no classes, inheritance, or this-capture)
- ✅ **DO use procedural/imperative style** as your default
- ✅ **DO borrow functional techniques** when they simplify code
- ✅ **DO avoid ideological purity** - the best paradigm is the one that fits the problem
- ❌ **NEVER add new Dockerfiles without user confirmation**
- ❌ **NEVER fetch initial data via AJAX on page load** - use server-side rendering instead
- ❌ **NEVER break progressive enhancement** - pages must work without JavaScript

**Note:** Go methods in the backend are fine and encouraged. This guideline applies to TypeScript/JavaScript frontend code only.**

### General
- ❌ **NEVER skip pre-commit hooks** unless explicitly requested
- ❌ **NEVER force push to main/master** branches
- ❌ **NEVER commit files with secrets** (.env, credentials.json, etc.)
- ❌ **NEVER make assumptions** - ask clarifying questions when uncertain
- ❌ **NEVER delete code without reading full context first** (minimum 20 lines before/after)
- ❌ **NEVER make cascading fix-up edits without git diff review**
  - After compilation error: STOP, review `git diff`, understand full impact
  - Use revert/reapply pattern instead of blind fixes
- ❌ **NEVER skip post-edit verification** - must compile after each file edit
- ❌ **NEVER run git commands concurrently** - always run sequentially:
  - Run `git add <files>` and wait for completion
  - Run `git commit -m "<message>"` and wait for completion
  - Run `git push` and wait for completion
  - Never use `&&` to chain git commands together
- ✅ **Make good organized git commits for the entire project (not just what you changed) and push - run git add, commit, push sequentially as separate commands, no need to repeatedly check status**
- ✅ **Indentation is always 2 spaces unless the language prohibits it**

### Cascading Fix-up Pattern (PROHIBITED)

**WHAT NOT TO DO** - This caused critical bugs:
```go
// ❌ WRONG: Blindly making fixes after compilation error

Edit 1: Delete deprecated code
[Compilation error: undefined Register]

Edit 2: Try to fix error (over-broad deletion)
[More errors: undefined Login, GetProfile, etc.]

Edit 3: Try to fix again (worse damage)
[Even more errors: major functionality broken]
```

**CORRECT APPROACH**:
```go
// ✅ CORRECT: Stop, understand, then fix deliberately

Edit 1: Delete deprecated code
[Compilation error: undefined Register]

STOP → Review git diff → Understand Register was accidentally deleted
RESTORE → Get exact Register function from git history
VERIFY → Compile successfully
```

**Key Principle**: When compilation errors occur after edits:
1. STOP - Don't make more edits
2. ANALYZE - Use `git diff` to understand what was changed
3. RECOVER - Restore what was accidentally deleted/broken
4. VERIFY - Compile and test before proceeding

---

## 🎯 CONTEXT-SPECIFIC RULES

### When Working on Frontend-Only Tasks
- **DO NOT touch backend code** - handlers, services, database layer
- **DO NOT modify API routes** - use existing endpoints only
- **DO NOT change database schema** - work with existing structure
- **If backend change seems necessary**:
  1. Identify the required change
  2. Explain why you need it
  3. Provide impact analysis
  4. **ASK FOR USER CONFIRMATION before proceeding**

### When Working on Full-Stack Tasks
- Backend changes are allowed when explicitly part of the task
- Still follow all database protocols (atomic changes, validation, etc.)
- **If modifying database schema:** Update local database after schema.sql changes (see Database Operations section)
- Still use Podman for all builds
- Still include Bruno tests for API changes

---

## ✅ MANDATORY REQUIREMENTS

### Database Operations (Full-Stack Tasks Only)
- ✅ Follow **pgx v5 standards** for all database operations
- ✅ Treat schema changes as **ATOMIC** - complete success or complete rejection
- ✅ **⚠️ CRITICAL: This is a pre-production application (NO production deployments exist)**
  - When `database/schema/schema.sql` is updated, local databases must be updated
  - **Option 1 (Recommended):** Recreate database with fresh schema:
    ```bash
    podman compose down -v  # Delete volumes (WARNING: loses all data)
    podman compose up -d    # Start fresh with new schema
    ```
  - **Option 2:** Manually apply schema changes to existing database using psql
  - **DO NOT create migration files** - no legacy schema support needed
- ✅ Use **pre-change checklist**: read schema → identify columns → plan changes → verify → read back
- ✅ **Post-change validation**: ensure schema.sql, models.go, and queries.sql are in sync

### Build & Deployment
- ✅ Use **Podman** exclusively (not Docker)
- ✅ All builds through existing **Dockerfile** and **docker-compose.yml**
- ✅ Stop building server binaries - everything goes through containers

### API Changes (Full-Stack Tasks Only)
- ✅ Include **Bruno OpenCollection YAML requests** with all API documentation
- ✅ Tests must be **comprehensive and cover three contexts**: no user, user, and admin
- ✅ Maintain backward compatibility for mobile apps and external consumers

### Frontend & Styling
- ✅ Always use **TailwindCSS classes** for all styling
- ✅ Convert all JavaScript to **TypeScript**
- ✅ **Never use Object-Oriented Programming** (no classes, inheritance, or this-capture)
- ✅ **Use procedural/imperative style** as your default
- ✅ **Borrow functional techniques** when they simplify code
- ✅ **Avoid ideological purity** - the best paradigm is the one that fits the problem
- ✅ **Extract domain concepts/types only when clearly beneficial** - apply YAGNI, avoid over-engineering
- ✅ **Render initial data server-side** in Go templates for fast page loads
- ✅ **Use JavaScript/HTMX for CRUD operations** (create, update, delete)
- ✅ **Ensure progressive enhancement** - pages work without JavaScript

### Service Layer Architecture
- ✅ **All business logic in services** - never in handlers
- ✅ **Services must be reusable** by both SSR handlers and API endpoints
- ✅ **Database operations through services only** - never direct from handlers
- ✅ **When adding features**: Add service logic → Create API endpoint → Use SSR for initial render → Use JS for updates

### Code Organization
- ✅ Minimize project structure changes
- ✅ Place new files in **contextually appropriate directories**
- ✅ Follow **KISS**, **DRY**, and **YAGNI** principles
- ✅ Use **multiple, logical git commits** with clear messages
- ❌ **NEVER duplicate types between handlers and templates** - define data types in handlers, reuse directly in templates
- ✅ **DO share handler types with templates** - templates should use handlers.CollectionData, handlers.BookInfo, etc. directly
- ❌ **NEVER create parallel type systems** - no templates.XxxData types for data that originates from handlers
- ✅ **DO enhance handler types with template-specific fields** when needed (e.g., add Icon, FormattedDate fields to handlers structs)
- ✅ **DO keep template-only types in templates package** - PageData, UnsafeHTML, and template-specific utilities are appropriate
- ❌ **NEVER create conversion helper functions** to map between handler and template types - use handler types directly

### Configuration & Environment
- ✅ If **.env is missing**, auto-generate secure values
- ✅ Never commit secrets to repository

### Code Modification Safety
- ✅ **Post-Edit Verification (MANDATORY for ALL file modifications)**:
  - Run `go build` for affected packages immediately after each edit
  - Review `git diff filename` to verify only intended changes
  - Validate functionality still works as expected
  - Never proceed to next file until current edit is verified
- ✅ **Backup Before Large Changes**:
  - Create a stash: `git stash push -m "Pre-cleanup snapshot"` before removing >50 lines
  - Or create a backup branch: `git branch backup-before-cleanup`
  - This allows instant recovery if mistakes occur
- ✅ **Large Deletion Safety Pattern**:
  - Read at least 20 lines before/after deletion target
  - Include unique identifiers in match (function signatures, specific comments)
  - Use narrow matches - avoid generic patterns
  - Verify line numbers match intended section

### Documentation

**Documentation Structure** (updated with full docs system):
- ✅ **README.md** - Project overview, quick start, and setup instructions only
- ✅ **docs/** - Comprehensive documentation system with search
- ✅ **docs/developer/api/** - API reference documentation (split by endpoint/category)
- ✅ **docs/user/** - User-facing features, guides, and workflows
- ✅ **docs/user/devices/** - Device setup guides (KOBO, KOReader, etc.)
- ✅ **docs/contributing/** - Development and contribution guides

**Where to document changes**:

| Change Type | Location | Examples |
|-------------|----------|----------|
| **User-facing features** | `docs/user/` | New features, UI changes, workflows |
| **API endpoints** | `docs/developer/api/<category>/<endpoint>.md` | New endpoints, modified responses, authentication changes |
| **API behavior** | Update existing `docs/developer/api/` files | Parameter changes, error codes, rate limits |
| **Device setup** | `docs/user/devices/` | New device support, setup instructions |
| **Development** | `docs/contributing/` | Build changes, architecture decisions |
| **Quick start/setup** | `README.md` | Installation, environment setup, first-run |
| **Breaking changes** | Both `README.md` and relevant `docs/` | Migration guides, deprecation notices |
| **Bug fixes** | Update relevant `docs/` only if user-visible | Clarifications, troubleshooting additions |
| **Bruno OpenCollection YAML tests** | `.yml` files in bruno folder in appropriate folder/sub-folder | API contract testing, examples |

**Documentation Update Workflow**:
1. **Identify the audience** (end users, developers, API consumers)
2. **Choose appropriate location** based on table above
3. **Update documentation** before or with code changes
4. **Verify documentation renders** at `/docs` endpoint
5. **Test search** finds new/updated content
6. **For API changes**: Update both `docs/developer/api/` files AND Bruno OpenCollection YAML `.yml` files
7. **Commit separately** with clear message: `docs: <description>`

**When in doubt**:
- End-user visible → `docs/user/`
- API reference → `docs/developer/api/`
- Setup/onboarding → `README.md`
- Development related → `docs/contributing/`

### Process & Continuity
- ✅ If mid-task and receive "no response", **continue the task**
- ✅ Verify no regressions before modifying/removing code

---

## 🔧 TECHNICAL STANDARDS

### Backend Stack
- **Language**: Go 1.25+
- **Database**: PostgreSQL 15+ with **pgx v5 driver** only
- **Authentication**: JWT tokens with bcrypt password hashing
- **Architecture**: Service layer pattern (handlers → services → database)

### Frontend Stack
- **Styling**: TailwindCSS (no custom CSS)
- **Language**: TypeScript (no JavaScript)
- **Templates**: HTMX with server-side rendering
- **Patterns**: Procedural/imperative with functional techniques where helpful (no OOP)

### Containerization
- **Runtime**: Podman (not Docker)
- **Build**: Existing Dockerfile and docker-compose.yml only
- **No local builds** allowed

---

## 🔄 ERROR RECOVERY PROTOCOL

When code modification mistakes occur (deleted wrong code, broke compilation, etc.):

### Immediate Actions
1. **STOP** - Don't make more edits
2. **ASSESS** - What was deleted? Is it critical?
3. **REVIEW** - Run `git diff` to see exact changes
4. **RESTORE** - Choose appropriate recovery method:
   - **Recent mistake**: `git checkout -- filename`
   - **Complex restoration**: Use `git show HEAD:filename` to recover deleted code
   - **Multiple files**: `git reset HEAD~1` (if safe)
5. **VERIFY** - Compile and test restored code
6. **DOCUMENT** - Note what went wrong for future reference

### Recovery Examples
```bash
# Recover a deleted function from original file
git show HEAD:internal/handlers/auth.go | sed -n '70,275p' > recovery.txt

# Revert entire file to original state
git checkout HEAD -- internal/handlers/auth.go

# Use git stash to save current state before rollback
git stash push -m "Broken state before fix"
git checkout -- internal/handlers/auth.go
```

### Prevention (Learn From Mistakes)
- Why did the mistake happen?
- Was it too-broad matching?
- Was it insufficient context reading?
- Was it cascading fix-up attempts?
- Update guidelines to prevent recurrence

---

## 📋 WORKFLOW CHECKLISTS

### Before Making Frontend-Only Changes
- [ ] Identify if backend modification could make implementation simpler
- [ ] Plan to use existing API endpoints only
- [ ] If backend change seems necessary, prepare confirmation request:
  - Required change description
  - Why it would help
  - Impact analysis
  - Alternative approaches considered
- [ ] Plan git commit structure (multiple logical commits)
- [ ] **Identify documentation location** (see Documentation section):
  - [ ] User-facing feature → `docs/user/`
  - [ ] UI/workflow changes → `docs/user/`
  - [ ] Setup instructions → `README.md`

### Before Making Full-Stack Changes
- [ ] Read current schema completely (if database changes)
- [ ] Identify all columns that must be preserved
- [ ] Plan exact changes needed
- [ ] Verify Podman will be used for builds
- [ ] Plan git commit structure (multiple logical commits)
- [ ] **Identify documentation location**:
  - [ ] API changes → `docs/developer/api/<category>/`
  - [ ] New endpoints → Create new `.md` file in `docs/developer/api/`
  - [ ] API behavior → Update existing `docs/developer/api/` files
  - [ ] Breaking changes → Both `README.md` + relevant `docs/`
  - [ ] Bruno OpenCollection YAML `.yml` files → Update/create alongside API changes

### During Schema Changes (Full-Stack Only)
- [ ] Read current schema completely
- [ ] Identify all columns that must be preserved
- [ ] Plan exact changes needed
- [ ] Set up verification step
- [ ] Make intended changes
- [ ] Immediately verify by reading back modified sections
- [ ] Confirm ALL expected columns are present
- [ ] Verify schema.sql, models.go, and queries.sql are in sync
- [ ] **Update local database** (choose ONE):
  - [ ] **Option 1 - Recreate database** (recommended, loses data):
    ```bash
    podman compose down -v  # Delete all volumes
    podman compose up -d    # Start with fresh schema
    ```
  - [ ] **Option 2 - Manual SQL migration** (preserves data):
    ```bash
    podman exec bookhoard_db psql -U postgres -d bookhoard -c "YOUR SQL HERE"
    ```
- [ ] Verify database has new schema (check column types, indexes, etc.)

### After API Changes
- [ ] Create/update Bruno OpenCollection YAML requests
- [ ] Test with no user context
- [ ] Test with regular user context
- [ ] Test with admin context
- [ ] Verify backward compatibility

### Before Committing
- [ ] **Run verification script**: `bash scripts/verify-guidelines.sh`
- [ ] **Fix any errors** - verification must pass (0 errors) to commit
- [ ] **Note warnings** - informational only, do not auto-fix
- [ ] Run tests: `go test ./... -v`
- [ ] Run lint/typecheck if available
- [ ] Ensure no secrets in changes
- [ ] Verify logical commit structure
- [ ] **Update documentation** (see Documentation section):
  - [ ] User-facing changes → `docs/user/`
  - [ ] API changes → `docs/developer/api/` + Bruno OpenCollection YAML `.yml` files
  - [ ] Setup/onboarding → `README.md`
  - [ ] Development changes → `docs/contributing/`
- [ ] **Verify docs render** at `/docs` endpoint
- [ ] **Test docs search** finds new content

### Error Recovery Protocol (If Code Mistakes Occur)
- [ ] **Stop immediately** - don't make more edits
- [ ] **Assess impact**: What was deleted? Is it critical?
- [ ] **Review git diff**: See exact changes made
- [ ] **Restore strategy**:
  - If recent mistake: `git checkout -- filename`
  - If complex: Reconstruct from git diff using `git show HEAD:filename`
- [ ] **Verify recovery**: Compile and test restored code
- [ ] **Document mistake**: Note what went wrong for future reference

### Phase Completion Verification (Before Declaring "Complete")
- [ ] All target code is removed/intact as intended
- [ ] No unintended code was deleted
- [ ] All affected files compile successfully
- [ ] Run `go build ./...` for entire project
- [ ] **Run verification script**: `bash scripts/verify-guidelines.sh` - must pass (0 errors)
- [ ] No critical functionality was broken
- [ ] Git diff shows only intended changes
- [ ] Review all modified files with `git diff --stat`

---

## 🏗 ARCHITECTURAL PATTERNS

### Current: Hybrid SSR
```
Browser → Go template (with data) → Display instantly
                    ↓
          JavaScript for interactivity (CRUD)
                    ↓
               Shared service layer
```

Ultimately, whenever you are unsure just ask for confirmation.