bookhoard/PROJECT_GUIDELINES.md

# Combined Project Guidelines for Bookhoard

## 🚨 CRITICAL PROHIBITIONS (Never violate these)

### Backend & Database

- ❌ **NEVER modify backend code when working on frontend-only tasks*
- ❌ **NEVER modify database schema** unless explicitly instructed for full-stack changes
- ❌ **NEVER use Docker** - use Podman only
- ❌ **NEVER build server binaries locally** - all builds through Dockerfile/docker-compose
- ❌ **NEVER create new migration files** - merge changes into current one until release
- ❌ **NEVER use `git checkout` on schema files** without checking what will be lost
- ❌ **NEVER break existing functionality** unless explicitly instructed
- ❌ **NEVER duplicate business logic** - keep logic in services, not handlers
- ❌ **NEVER bypass service layer** - all database operations must go through services

### Testing

- ✅ **ALWAYS use `setupTestServer()` helper from `cmd/server/tests/test_helpers_test.go`*
- ✅ **Share one test setup across all subtests** - call `setupTestServer()` once at test function level, not per subtest
- ✅ **Prefer table-driven tests** - use `t.Run()` with test cases instead of duplicate test functions
- ✅ **Configure database pools efficiently** - use `max_conns=1` for test pools (via `pgxpool.ParseConfig()`) to prevent connection exhaustion
- ❌ **NEVER create separate `pgxpool` per test** - each pool creates 4 connections by default; 78 tests = 312 potential connections > PostgreSQL's 100 limit
- ❌ **NEVER call `setupTestServer()` in loops or within subtests** - creates unnecessary database pools and exhausts connections
- ✅ **DO verify tests pass** - run full test suite before completing work
- ✅ **Use `t.Cleanup()` properly** - the `TestServerSetup` pattern automatically handles cleanup via `t.Cleanup()`

### Frontend & Styling

- ❌ **NEVER modify backend/API for frontend features without user confirmation*
- ❌ **NEVER use custom CSS** - TailwindCSS classes only
  - **⚠️ EXCEPTION**: `templates/error.templ` may have inline CSS because error pages must work when main app fails (404, server errors, CSS fails to load)
- ❌ **NEVER use JavaScript** - convert all to TypeScript
  - **⚠️ EXCEPTION**: Inline JS function calls in HTML attributes (e.g., `onclick="myFunction()"`) are acceptable for simple interactions
- ❌ **NEVER use Object-Oriented Programming** (no classes, inheritance, or this-capture)
- ✅ **DO use procedural/imperative style** as your default
- ✅ **DO borrow functional techniques** when they simplify code
- ✅ **DO avoid ideological purity** - the best paradigm is the one that fits the problem
- ❌ **NEVER add new Dockerfiles without user confirmation*
- ❌ **NEVER fetch initial data via AJAX on page load** - use server-side rendering instead
- ❌ **NEVER break progressive enhancement** - pages must work without JavaScript

**Note:** Go methods in the backend are fine and encouraged. This guideline applies to TypeScript/JavaScript frontend code only.

### General

- ❌ **NEVER skip pre-commit hooks** unless explicitly requested
- ❌ **NEVER force push to main/master** branches
- ❌ **NEVER commit files with secrets** (.env, credentials.json, etc.)
- ❌ **NEVER make assumptions** - ask clarifying questions when uncertain
- ❌ **NEVER delete code without reading full context first** (minimum 20 lines before/after)
- ❌ **NEVER make cascading fix-up edits without git diff review*
  - After compilation error: STOP, review `git diff`, understand full impact
  - Use revert/reapply pattern instead of blind fixes
- ❌ **NEVER skip post-edit verification** - must compile after each file edit
- ❌ **NEVER run git commands concurrently** - always run sequentially:
  - Run `git add <files>` and wait for completion
  - Run `git commit -m "<message>"` and wait for completion
  - Run `git push` and wait for completion
  - Never use `&&` to chain git commands together
- ✅ **Make good organized git commits for the entire project (not just what you changed) and push - run git add, commit, push sequentially as separate commands, no need to repeatedly check status*
- ✅ **Indentation is always 2 spaces unless the language prohibits it*

### Cascading Fix-up Pattern (PROHIBITED)

**WHAT NOT TO DO** - This caused critical bugs:

```go
// ❌ WRONG: Blindly making fixes after compilation error

Edit 1: Delete deprecated code
[Compilation error: undefined Register]

Edit 2: Try to fix error (over-broad deletion)
[More errors: undefined Login, GetProfile, etc.]

Edit 3: Try to fix again (worse damage)
[Even more errors: major functionality broken]
```

**CORRECT APPROACH**:

```go
// ✅ CORRECT: Stop, understand, then fix deliberately

Edit 1: Delete deprecated code
[Compilation error: undefined Register]

STOP → Review git diff → Understand Register was accidentally deleted
RESTORE → Get exact Register function from git history
VERIFY → Compile successfully
```

**Key Principle**: When compilation errors occur after edits:

1. STOP - Don't make more edits
2. ANALYZE - Use `git diff` to understand what was changed
3. RECOVER - Restore what was accidentally deleted/broken
4. VERIFY - Compile and test before proceeding

---

## 🎯 CONTEXT-SPECIFIC RULES

### When Working on Frontend-Only Tasks

- **DO NOT touch backend code** - handlers, services, database layer
- **DO NOT modify API routes** - use existing endpoints only
- **DO NOT change database schema** - work with existing structure
- **If backend change seems necessary**:
  1. Identify the required change
  2. Explain why you need it
  3. Provide impact analysis
  4. **ASK FOR USER CONFIRMATION before proceeding*

### When Working on Full-Stack Tasks

- Backend changes are allowed when explicitly part of the task
- Still follow all database protocols (atomic changes, validation, etc.)
- **If modifying database schema:** Update local database after schema.sql changes (see Database Operations section)
- Still use Podman for all builds
- Still include Bruno requests for API changes

---

## ✅ MANDATORY REQUIREMENTS

### Database Operations (Full-Stack Tasks Only)

- ✅ Follow **pgx v5 standards** for all database operations
- ✅ Treat schema changes as **ATOMIC** - complete success or complete rejection
- ✅ **⚠️ CRITICAL: This is a pre-production application (NO production deployments exist)*
  - When `database/schema/schema.sql` is updated, local databases must be updated
  - **Option 1 (Recommended):** Recreate database with fresh schema:
    ```bash
    podman compose down -v  # Delete volumes (WARNING: loses all data)
    podman compose up -d    # Start fresh with new schema
    ```
  - **Option 2:** Manually apply schema changes to existing database using psql
  - **DO NOT create migration files** - no legacy schema support needed
- ✅ Use **pre-change checklist**: read schema → identify columns → plan changes → verify → read back
- ✅ **Post-change validation**: ensure schema.sql, models.go, and queries.sql are in sync

### Build & Deployment

- ✅ Use **Podman** exclusively (not Docker)
- ✅ All builds through existing **Dockerfile** and **docker-compose.yml*
- ✅ Stop building server binaries - everything goes through containers

### API Changes (Full-Stack Tasks Only)

- ✅ Include **Bruno OpenCollection YAML requests** with all API documentation
- ✅ Integration Tests (cmd/server/tests) must be **comprehensive and cover three contexts**: no user, user, and admin
- ✅ Maintain backward compatibility for mobile apps and external consumers

### Frontend & Styling

- ✅ Always use **TailwindCSS classes** for all styling
- ✅ Convert all JavaScript to **TypeScript**
- ✅ **Never use Object-Oriented Programming** (no classes, inheritance, or this-capture)
- ✅ **Use procedural/imperative style** as your default
- ✅ **Borrow functional techniques** when they simplify code
- ✅ **Avoid ideological purity** - the best paradigm is the one that fits the problem
- ✅ **Extract domain concepts/types only when clearly beneficial** - apply YAGNI, avoid over-engineering
- ✅ **Render initial data server-side** in Go templates for fast page loads
- ✅ **Use JavaScript/HTMX for CRUD operations** (create, update, delete)
- ✅ **Ensure progressive enhancement** - pages work without JavaScript

### Service Layer Architecture

- ✅ **All business logic in services** - never in handlers
- ✅ **Services must be reusable** by both SSR handlers and API endpoints
- ✅ **Database operations through services only** - never direct from handlers
- ✅ **When adding features**: Add service logic → Create API endpoint → Use SSR for initial render → Use JS for updates

### Code Organization

- ✅ Minimize project structure changes
- ✅ Place new files in **contextually appropriate directories*
- ✅ Follow **KISS**, **DRY**, and **YAGNI** principles
- ✅ Use **multiple, logical git commits** with clear messages
- ❌ **NEVER duplicate types between handlers and templates** - define data types in handlers, reuse directly in templates
- ✅ **DO share handler types with templates** - templates should use handlers.CollectionData, handlers.BookInfo, etc. directly
- ❌ **NEVER create parallel type systems** - no templates.XxxData types for data that originates from handlers
- ✅ **DO enhance handler types with template-specific fields** when needed (e.g., add Icon, FormattedDate fields to handlers structs)
- ✅ **DO keep template-only types in templates package** - PageData, UnsafeHTML, and template-specific utilities are appropriate
- ❌ **NEVER create conversion helper functions** to map between handler and template types - use handler types directly

### Configuration & Environment

- ✅ If **.env is missing**, auto-generate secure values
- ✅ Never commit secrets to repository

### Code Modification Safety

- ✅ **Post-Edit Verification (MANDATORY for ALL file modifications)**:
  - Run `go build` for affected packages immediately after each edit
  - Review `git diff filename` to verify only intended changes
  - Validate functionality still works as expected
  - Never proceed to next file until current edit is verified
- ✅ **Backup Before Large Changes**:
  - Create a stash: `git stash push -m "Pre-cleanup snapshot"` before removing >50 lines
  - Or create a backup branch: `git branch backup-before-cleanup`
  - This allows instant recovery if mistakes occur
- ✅ **Large Deletion Safety Pattern**:
  - Read at least 20 lines before/after deletion target
  - Include unique identifiers in match (function signatures, specific comments)
  - Use narrow matches - avoid generic patterns
  - Verify line numbers match intended section

### Documentation

**Documentation Structure** (updated with full docs system):

- ✅ **README.md** - Project overview, quick start, and setup instructions only
- ✅ **docs/** - Comprehensive documentation system with search
- ✅ **docs/developer/api/** - API reference documentation (split by endpoint/category)
- ✅ **docs/user/** - User-facing features, guides, and workflows
- ✅ **docs/user/devices/** - Device setup guides (KOBO, KOReader, etc.)
- ✅ **docs/contributing/** - Development and contribution guides

**Where to document changes**:

| Change Type                         | Location                                                      | Examples                                                  |
| ----------------------------------- | ------------------------------------------------------------- | --------------------------------------------------------- |
| **User-facing features**            | `docs/user/`                                                  | New features, UI changes, workflows                       |
| **API endpoints**                   | `docs/developer/api/<category>/<endpoint>.md`                 | New endpoints, modified responses, authentication changes |
| **API behavior**                    | Update existing `docs/developer/api/` files                   | Parameter changes, error codes, rate limits               |
| **Device setup**                    | `docs/user/devices/`                                          | New device support, setup instructions                    |
| **Development**                     | `docs/contributing/`                                          | Build changes, architecture decisions                     |
| **Quick start/setup**               | `README.md`                                                   | Installation, environment setup, first-run                |
| **Breaking changes**                | Both `README.md` and relevant `docs/`                         | Migration guides, deprecation notices                     |
| **Bug fixes**                       | Update relevant `docs/` only if user-visible                  | Clarifications, troubleshooting additions                 |
| **Bruno OpenCollection YAML requests** | `.yml` files in bruno folder in appropriate folder/sub-folder | API contract testing, examples                            |

**Documentation Update Workflow**:

1. **Identify the audience** (end users, developers, API consumers)
2. **Choose appropriate location** based on table above
3. **Update documentation** before or with code changes
4. **Verify documentation renders** at `/docs` endpoint
5. **Test search** finds new/updated content
6. **For API changes**: Update both `docs/developer/api/` files, integration test files (cmd/server/tests) AND Bruno OpenCollection YAML `.yml` files
7. **Commit separately** with clear message: `docs: <description>`

**When in doubt**:

- End-user visible → `docs/user/`
- API reference → `docs/developer/api/`
- Setup/onboarding → `README.md`
- Development related → `docs/contributing/`

### Process & Continuity

- ✅ If mid-task and receive "no response", **continue the task*
- ✅ Verify no regressions before modifying/removing code

---

## 🔧 TECHNICAL STANDARDS

### Backend Stack

- **Language**: Go 1.25+
- **Database**: PostgreSQL 15+ with **pgx v5 driver** only
- **Authentication**: JWT tokens with bcrypt password hashing
- **Architecture**: Service layer pattern (handlers → services → database)

### Frontend Stack

- **Rendering**: SSR-first - Go templates render initial page with all data
- **Interactivity**: Alpine.js for UI state (modals, dropdowns, transitions)
- **Dynamic Updates**: HTMX for CRUD operations (no full page reloads)
- **Styling**: TailwindCSS (no custom CSS)
- **Language**: TypeScript (no JavaScript)
- **Patterns**: Procedural/imperative with functional techniques where helpful (no OOP)

### SSR-First Principles

- ❌ **NEVER fetch initial data via AJAX** - server renders complete page
- ❌ **NEVER fetch data in Alpine x-init** - data already SSR'd
- ✅ **Use Alpine.js only for UI state** - modal visibility, dropdown toggles
- ✅ **Use HTMX for dynamic operations** - form submissions, partial page updates
- ✅ **x-init is for setup only** - event listeners, not data fetching
- ✅ **Data fetching happens after user actions** - not on page load

### Page Type Classifications

1. **Type 1: 80% SSR (most pages)**
   - Backend provides all initial data
   - Alpine handles modals/dropdowns only
   - x-init NEVER fetches data

2. **Type 2: SSR + Interactive (dashboard, bookshelf)**
   - Backend provides initial data
   - Alpine handles interactivity (drag-drop, filtering)
   - x-init ONLY sets up event listeners

3. **Type 3: 80% TypeScript (analytics, complex dashboards)**
   - Some client-side data fetching acceptable
   - Still prefer SSR when possible

### Containerization

- **Runtime**: Podman (not Docker)
- **Build**: Existing Dockerfile and docker-compose.yml only
- **No local builds** allowed

---

## 🔄 ERROR RECOVERY PROTOCOL

When code modification mistakes occur (deleted wrong code, broke compilation, etc.):

### Immediate Actions

1. **STOP** - Don't make more edits
2. **ASSESS** - What was deleted? Is it critical?
3. **REVIEW** - Run `git diff` to see exact changes
4. **RESTORE** - Choose appropriate recovery method:
   - **Recent mistake**: `git checkout -- filename`
   - **Complex restoration**: Use `git show HEAD:filename` to recover deleted code
   - **Multiple files**: `git reset HEAD~1` (if safe)
5. **VERIFY** - Compile and test restored code
6. **DOCUMENT** - Note what went wrong for future reference

### Recovery Examples

```bash
# Recover a deleted function from original file
git show HEAD:internal/handlers/auth.go | sed -n '70,275p' > recovery.txt

# Revert entire file to original state
git checkout HEAD -- internal/handlers/auth.go

# Use git stash to save current state before rollback
git stash push -m "Broken state before fix"
git checkout -- internal/handlers/auth.go
```

### Prevention (Learn From Mistakes)

- Why did the mistake happen?
- Was it too-broad matching?
- Was it insufficient context reading?
- Was it cascading fix-up attempts?
- Update guidelines to prevent recurrence

---

## 📋 WORKFLOW CHECKLISTS

### Before Making Frontend-Only Changes

- [ ] Identify if backend modification could make implementation simpler
- [ ] Plan to use existing API endpoints only
- [ ] If backend change seems necessary, prepare confirmation request:
  - Required change description
  - Why it would help
  - Impact analysis
  - Alternative approaches considered
- [ ] Plan git commit structure (multiple logical commits)
- [ ] **Identify documentation location** (see Documentation section):
  - [ ] User-facing feature → `docs/user/`
  - [ ] UI/workflow changes → `docs/user/`
  - [ ] Setup instructions → `README.md`

### Before Making Full-Stack Changes

- [ ] Read current schema completely (if database changes)
- [ ] Identify all columns that must be preserved
- [ ] Plan exact changes needed
- [ ] Verify Podman will be used for builds
- [ ] Plan git commit structure (multiple logical commits)
- [ ] **Identify documentation location**:
  - [ ] API changes → `docs/developer/api/<category>/`
  - [ ] New endpoints → Create new `.md` file in `docs/developer/api/`
  - [ ] API behavior → Update existing `docs/developer/api/` files
  - [ ] Breaking changes → Both `README.md` + relevant `docs/`
  - [ ] Bruno OpenCollection YAML `.yml` files → Update/create alongside API changes
  - [ ] Integration tests (cmd/server/tests) → Update/create alongside API changes

### During Schema Changes (Full-Stack Only)

- [ ] Read current schema completely
- [ ] Identify all columns that must be preserved
- [ ] Plan exact changes needed
- [ ] Set up verification step
- [ ] Make intended changes
- [ ] Immediately verify by reading back modified sections
- [ ] Confirm ALL expected columns are present
- [ ] Verify schema.sql, models.go, and queries.sql are in sync
- [ ] **Update local database** (choose ONE):
  - [ ] **Option 1 - Recreate database** (recommended, loses data):
    ```bash
    podman compose down -v  # Delete all volumes
    podman compose up -d    # Start with fresh schema
    ```
  - [ ] **Option 2 - Manual SQL migration** (preserves data):
    ```bash
    podman exec bookhoard_db psql -U postgres -d bookhoard -c "YOUR SQL HERE"
    ```
- [ ] Verify database has new schema (check column types, indexes, etc.)

### After API Changes

- [ ] Create/update Bruno OpenCollection YAML requests
- [ ] Create/update Integration tests (cmd/server/tests) 
- [ ] Test with no user context
- [ ] Test with regular user context
- [ ] Test with admin context
- [ ] Verify backward compatibility

### Before Committing

- [ ] **Run verification script**: `bash scripts/verify-guidelines.sh`
- [ ] **Fix any errors** - verification must pass (0 errors) to commit
- [ ] **Note warnings** - informational only, do not auto-fix
- [ ] Run tests: `go test ./... -v`
- [ ] Run lint/typecheck if available
- [ ] Ensure no secrets in changes
- [ ] Verify logical commit structure
- [ ] **Update documentation** (see Documentation section):
  - [ ] User-facing changes → `docs/user/`
  - [ ] API changes → `docs/developer/api/` + Integration tests + Bruno OpenCollection YAML `.yml` files
  - [ ] Setup/onboarding → `README.md`
  - [ ] Development changes → `docs/contributing/`
- [ ] **Verify docs render** at `/docs` endpoint
- [ ] **Test docs search** finds new content

### Error Recovery Protocol (If Code Mistakes Occur)

- [ ] **Stop immediately** - don't make more edits
- [ ] **Assess impact**: What was deleted? Is it critical?
- [ ] **Review git diff**: See exact changes made
- [ ] **Restore strategy**:
  - If recent mistake: `git checkout -- filename`
  - If complex: Reconstruct from git diff using `git show HEAD:filename`
- [ ] **Verify recovery**: Compile and test restored code
- [ ] **Document mistake**: Note what went wrong for future reference

### Phase Completion Verification (Before Declaring "Complete")

- [ ] All target code is removed/intact as intended
- [ ] No unintended code was deleted
- [ ] All affected files compile successfully
- [ ] Run `go build ./...` for entire project
- [ ] **Run verification script**: `bash scripts/verify-guidelines.sh` - must pass (0 errors)
- [ ] No critical functionality was broken
- [ ] Git diff shows only intended changes
- [ ] Review all modified files with `git diff --stat`

---

## 🏗 ARCHITECTURAL PATTERNS

### Current: SSR-First with Alpine.js + HTMX

```
Browser → Go template (SSR with all data) → Display instantly
                      ↓
        Alpine.js for UI state (modals, dropdowns)
                      ↓
                 HTMX for CRUD (forms, updates)
                      ↓
           Shared service layer (Go)
```

**Layer Responsibilities:**

| Layer | Responsibility |
|-------|---------------|
| **Go Template** | SSR initial page with real data |
| **Alpine.js** | UI state only (x-data, x-show, transitions) |
| **HTMX** | Dynamic updates without page reload |
| **TypeScript** | Pure business logic (API calls, data processing) |

**See also:** `SSR_FIRST_ALPINE_GUIDE.md` and `ALPINE_COMPLETION_GUIDE.md`

Ultimately, whenever you are unsure just ask for confirmation.