From d906ef3e4da86b94a6126bb7ca1cd4e10670f254 Mon Sep 17 00:00:00 2001 From: John O'Keefe Date: Sun, 1 Feb 2026 13:21:37 -0500 Subject: [PATCH] docs: add testing guide and progress routes analysis documentation --- docs/PROGRESS_ROUTES_ANALYSIS.md | 423 +++++++++++++++++++++++ docs/TESTING.md | 553 +++++++++++++++++++++++++++++++ 2 files changed, 976 insertions(+) create mode 100644 docs/PROGRESS_ROUTES_ANALYSIS.md create mode 100644 docs/TESTING.md diff --git a/docs/PROGRESS_ROUTES_ANALYSIS.md b/docs/PROGRESS_ROUTES_ANALYSIS.md new file mode 100644 index 0000000..2ecf5d0 --- /dev/null +++ b/docs/PROGRESS_ROUTES_ANALYSIS.md @@ -0,0 +1,423 @@ +# Progress Routes Analysis & Thoughts + +## Overview + +This document explores the current state of progress tracking in Bookmann, the migration from legacy media-item-specific routes to universal cross-device progress, and considerations for the future. + +--- + +## Current State + +### Legacy Routes (Marked as Deprecated) + +Located in `internal/handlers/ebook.go:114-117`: + +```go +// Legacy progress routes (deprecated - use universal progress instead) +g.GET("/api/media-items/:id/progress", h.GetMediaReadingProgress) +g.PUT("/api/media-items/:id/progress", h.UpdateMediaReadingProgress) +g.DELETE("/api/media-items/:id/progress", h.DeleteMediaReadingProgress) +``` + +**Purpose**: These routes handle progress tracking for a specific media item from the `media_items` table. + +**Data Source**: Likely queries the `reading_progress` table filtered by `media_item_id`. + +**Current Status**: Explicitly marked as "legacy" and "deprecated" in code comments. + +--- + +### Universal Progress Routes (Phase 1 Implementation) + +Located in `internal/handlers/ebook.go:119-122`: + +```go +// Universal Progress routes (Phase 1) +g.GET("/api/progress/:id", h.GetUniversalProgress) +g.POST("/api/progress/:id", h.UpdateUniversalProgress) +g.GET("/api/progress/:id/history", h.GetProgressHistory) +``` + +**Purpose**: These routes provide "universal" progress tracking that works across devices and media types. + +**Data Source**: Uses enhanced `reading_progress` table with additional fields: +- `percentage` - Universal percentage (0-1) +- `character_offset` - Character-based positioning +- `epubcfi` - EPUB Canonical Fragment Identifier +- `chapter` + `chapter_progress` - Chapter-based tracking +- Viewport coordinates (viewport_x, viewport_y, zoom_level) +- Scroll positions (scroll_position_x, scroll_position_y) +- Panel number for comics/manga +- Reading mode indicator + +**Device Sync Metadata**: +- `last_sync_device` - Which device last updated +- `last_sync_source` - Source type (koreader, kobo, web, etc.) +- `last_sync_timestamp` - When sync occurred +- `conflict_detected` - Boolean flag for conflicts +- `conflict_resolved` - Boolean flag for resolution status + +--- + +## Why the Migration Happened + +### 1. **Cross-Platform Kindle Ecosystem Vision** + +Bookmann aims to replace the Kindle ecosystem, which requires: +- Syncing progress across multiple devices (Kindle, Kobo, phone, web) +- Handling different progress formats (page numbers, percentages, CFI, character offsets) +- Maintaining reading state across different device types +- Supporting offline reading with sync queues + +### 2. **Format Diversity** + +Different e-readers and formats use different progress indicators: + +| Format/Device | Progress Type | Example | +|---------------|---------------|---------| +| EPUB (KOReader) | EPUBCFI | `epubcfi(/6/4[chap1ref]!/4/2/1:0)` | +| EPUB (Kobo) | Page # + Total | `page 234 of 456` | +| PDF | Page # | `page 45` | +| Web Reader | Percentage | `0.45 (45%)` | +| TXT/Mobi | Character Offset | `offset 12345` | +| Comics/Manga | Panel # | `panel 7` | +| Kindle | Location # | `location 1234` | + +The legacy `media-items/:id/progress` routes couldn't handle this diversity. + +### 3. **Device Sync Architecture** + +Universal progress enables: +- Real-time sync via WebSocket (`/ws/sync`) +- Offline queue support (`/api/queue/*`) +- Conflict detection and resolution +- Checkpoint mode for battery optimization +- Progress history tracking + +--- + +## Current Database Schema + +From `database/schema/schema.sql:130-158`: + +```sql +CREATE TABLE reading_progress ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + media_item_id UUID NOT NULL REFERENCES media_items(id) ON DELETE CASCADE, + user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE, + + -- Legacy fields + current_page INTEGER DEFAULT 0, + total_pages INTEGER, + last_read_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), + + -- Universal Progress Tracking (Phase 1) + percentage FLOAT CHECK (percentage >= 0 AND percentage <= 1), + character_offset BIGINT, + epubcfi TEXT, + chapter INTEGER, + chapter_progress FLOAT CHECK (chapter_progress >= 0 AND chapter_progress <= 1), + viewport_x FLOAT DEFAULT 0, + viewport_y FLOAT DEFAULT 0, + zoom_level FLOAT DEFAULT 1.0, + scroll_position_x FLOAT DEFAULT 0, + scroll_position_y FLOAT DEFAULT 0, + panel_number INTEGER, + reading_mode VARCHAR(20), + + -- Device Sync Metadata (Phase 1) + last_sync_device VARCHAR(50), + last_sync_source VARCHAR(20), + last_sync_timestamp TIMESTAMP WITH TIME ZONE, + conflict_detected BOOLEAN DEFAULT FALSE, + conflict_resolved BOOLEAN DEFAULT TRUE, + + UNIQUE(media_item_id, user_id) +); +``` + +**Backward Compatibility View** (Line 160-168): + +```sql +CREATE VIEW ebook_reading_progress AS +SELECT rp.*, + mi.id as ebook_id -- Map media_item_id to ebook_id for compatibility +FROM reading_progress rp +JOIN media_items mi ON rp.media_item_id = mi.id +JOIN libraries l ON mi.library_id = l.id +JOIN library_types lt ON l.library_type_id = lt.id +WHERE lt.name = 'ebooks'; +``` + +--- + +## The Migration Challenge + +### Issue: Two Parallel Systems + +Currently, **both** systems exist side-by-side: + +1. **Legacy routes** (`/api/media-items/:id/progress`) + - Likely use simple `current_page` / `total_pages` fields + - Media-item scoped + - No device sync metadata + +2. **Universal routes** (`/api/progress/:id`) + - Use rich progress tracking with multiple formats + - Device-aware + - Include sync metadata + +### Question: What Does `:id` Mean? + +**Legacy**: `:id` = `media_item_id` (UUID of the book) + +**Universal**: `:id` = ??? (Could be same media_item_id, or could be a different identifier) + +**Ambiguity**: The routes use the same parameter name but might mean different things. + +### Potential Problems + +1. **Data Duplication**: If both systems write to `reading_progress` table, they might overwrite each other +2. **Client Confusion**: Which endpoint should clients use? +3. **Migration Path**: How do existing clients using legacy endpoints transition? +4. **API Consistency**: Having two different endpoints for similar functionality is confusing + +--- + +## Observations & Concerns + +### 1. **Incomplete Migration** + +The legacy routes are marked as deprecated but **still active**. This suggests: +- Migration is ongoing, not complete +- Some clients might still depend on legacy routes +- Fear of breaking existing integrations + +### 2. **Backward Compatibility View** + +The `ebook_reading_progress` view exists to maintain compatibility with the old `ebooks` table. This adds: +- Query overhead (JOINs to filter by library type) +- Developer confusion (which table/view to query?) +- Technical debt (maintaining two ways to access data) + +### 3. **Route Naming Inconsistency** + +- Legacy: `/api/media-items/:id/progress` (RESTful, nested under media-item) +- Universal: `/api/progress/:id` (flat structure, not nested) + +**Question**: Should universal progress be under `/api/media-items/:id/universal-progress` for consistency? + +### 4. **HTTP Method Mismatch** + +Legacy routes use: +- `PUT /api/media-items/:id/progress` (update progress) + +Universal routes use: +- `POST /api/progress/:id` (update progress) + +**REST convention**: `PUT` is idempotent, `POST` is not. For progress updates, `PUT` might be more appropriate since setting the same progress twice should have the same effect. + +### 5. **Missing Delete Operation** + +Universal routes don't have a `DELETE /api/progress/:id` endpoint. Legacy does: +- `DELETE /api/media-items/:id/progress` (clear progress) + +**Question**: Should there be a way to reset progress via universal routes? + +--- + +## Potential Future Directions + +### Option 1: Full Migration (Clean Break) + +**Action**: Remove all legacy routes and views. + +**Steps**: +1. Deprecate legacy routes in API documentation (return `Warning` header) +2. Add a 6-month migration timeline +3. Remove `/api/media-items/:id/progress` routes +4. Drop `ebook_reading_progress` view +5. Update all clients to use universal routes + +**Pros**: +- Cleaner API surface +- Single source of truth +- Less maintenance burden +- Clearer documentation + +**Cons**: +- Breaking change for existing clients +- Mobile apps might need updates +- External integrations could break + +### Option 2: Compatibility Layer (Adapter Pattern) + +**Action**: Keep legacy routes but make them thin wrappers around universal routes. + +**Implementation**: +```go +// Legacy route calls universal route internally +func (h *Handler) GetMediaReadingProgress(c echo.Context) error { + mediaItemID := c.Param("id") + // Extract user_id from JWT + // Call h.GetUniversalProgress with same IDs + // Transform response if needed +} +``` + +**Pros**: +- No breaking changes +- Gradual migration path +- Single implementation (universal routes) + +**Cons**: +- Maintains API surface area +- Slight performance overhead (function call) +- Still confusing to have two endpoints + +### Option 3: Unified Endpoint (Best of Both) + +**Action**: Create a single endpoint that handles both use cases. + +**Proposed**: +``` +GET /api/media-items/:id/progress?format=universal +PUT /api/media-items/:id/progress?format=universal +DELETE /api/media-items/:id/progress +``` + +The `format` query parameter determines: +- `format=simple` (default): Returns basic page/percentage (legacy behavior) +- `format=universal`: Returns full device-aware progress with metadata + +**Pros**: +- Single endpoint +- Backward compatible +- Clear migration path via query parameter +- RESTful structure (nested under media-items) + +**Cons**: +- More complex handler logic +- Need to maintain both formats in response + +### Option 4: Versioned API (Cleanest Long-Term) + +**Action**: Use API versioning to separate old and new. + +**Proposed**: +``` +# v1 (Legacy) +GET /api/v1/media-items/:id/progress +PUT /api/v1/media-items/:id/progress +DELETE /api/v1/media-items/:id/progress + +# v2 (Universal) +GET /api/v2/media-items/:id/progress +PUT /api/v2/media-items/:id/progress +GET /api/v2/media-items/:id/progress/history +``` + +**Pros**: +- Clean separation +- Can deprecate v1 independently +- Standard industry practice +- Clear migration documentation + +**Cons**: +- Need to implement version routing +- More upfront work +- Maintenance of two versions temporarily + +--- + +## Unanswered Questions for Discussion + +1. **Are any clients currently using the legacy progress routes?** + - If yes, which ones? (mobile app, web app, third-party integrations?) + - Can they be updated easily? + +2. **What does the `:id` parameter represent in universal progress routes?** + - Is it still `media_item_id`? + - Or is it a `reading_progress` record ID? + - Need to check implementation to confirm + +3. **Why was `/api/progress/:id` chosen instead of `/api/media-items/:id/universal-progress`?** + - Flat structure vs nested structure design decision + - Might indicate plans for progress to exist independently of media items? + +4. **Is the legacy route implementation actually different, or just deprecated?** + - Need to read the handler implementations to compare + - They might be calling the same underlying code + +5. **Should we maintain progress deletion functionality?** + - Universal routes don't have DELETE + - Is deleting progress a necessary feature? + +6. **What's the timeline for removing legacy routes?** + - Already marked deprecated, but when can we delete them? + - Need to coordinate with mobile app releases + +7. **How does the backward compatibility view affect performance?** + - The `ebook_reading_progress` view requires JOINs + - Is it used anywhere, or can it be dropped? + +--- + +## Recommendations + +### Immediate Actions (Discussion Phase) + +1. **Audit Current Usage** + - Search codebase for references to legacy routes + - Check if any external documentation mentions these endpoints + - Identify all clients (web, mobile, third-party) + +2. **Compare Implementations** + - Read handler code for both legacy and universal routes + - Document differences in behavior + - Determine if they're truly different or just deprecated wrappers + +3. **Clarify API Contract** + - Define what `:id` means in universal routes + - Document expected request/response formats + - Add examples for different device types + +4. **Performance Analysis** + - Query database to see how many records use legacy fields vs universal + - Check if backward compatibility view is actually used + - Benchmark query performance with/without views + +### Future Considerations + +1. **Choose a Migration Strategy** + - Review Options 1-4 above + - Consider breaking changes vs compatibility + - Plan timeline based on client usage + +2. **API Versioning Decision** + - Decide if we want `/api/v1/` and `/api/v2/` structure + - Or use different approach (headers, content negotiation) + +3. **Documentation Updates** + - Update API_REFERENCE.md with clear deprecation notices + - Add migration guide for clients + - Document best practices for progress tracking + +4. **Test Coverage** + - Ensure both legacy and universal routes have comprehensive tests + - Add integration tests for cross-device sync scenarios + - Test conflict resolution workflows + +--- + +## Next Steps for Discussion + +1. **Review handler implementations** to understand actual differences +2. **Check client usage** (web app, mobile apps, Bruno tests) +3. **Decide on migration timeline** and breaking change tolerance +4. **Choose unified strategy** (Options 1-4 or hybrid) +5. **Plan implementation** with backward compatibility in mind + +--- + +*Document created for future discussion. No changes to be made without review.* diff --git a/docs/TESTING.md b/docs/TESTING.md new file mode 100644 index 0000000..a80f303 --- /dev/null +++ b/docs/TESTING.md @@ -0,0 +1,553 @@ +# Bookmann Integration Test Suite Documentation + +## Overview + +This document provides comprehensive information about the integration test suite for Bookmann, including how to run tests, what they cover, and best practices for adding new tests. + +## Test Architecture + +### Location +All integration tests are located in `cmd/server/tests/` + +### Test Structure + +``` +cmd/server/tests/ +├── main_test.go # Framework verification +├── setup_test.go # Test setup and helper functions +├── test_helpers.go # Reusable test helpers +├── testrunner_test.go # Test runner verification +│ +├── analytics_test.go # Analytics endpoints (NEW) +├── auth_test.go # Authentication & authorization +├── book_matching_test.go # Book matching & bulk linking (NEW) +├── collections_bulk_test.go # Bulk collection operations (NEW) +├── conflicts_bulk_test.go # Bulk conflict resolution (NEW) +├── conflicts_test.go # Conflict management +├── device_cap_test.go # Device capability tests +├── device_test.go # Device management +├── edge_cases_test.go # Edge case coverage +├── filtering_test.go # Filtering functionality +├── isbn_and_library_test.go # ISBN & library tests +├── kobo_test.go # Kobo device sync +├── koreader_test.go # KOReader sync +├── library_test.go # Library management +├── library_test_comprehensive.go # Comprehensive library tests +├── media_bulk_test.go # Bulk media operations (NEW) +├── new_fixes_test.go # Recent fixes validation +├── opds_test.go # OPDS endpoints (NEW) +├── phase1_integration_test.go # Phase 1 integration tests +├── queue_test.go # Sync queue management +├── refresh_token_test.go # Token refresh flow (NEW) +├── registration_test.go # Device registration flow +├── search_test.go # Search functionality +├── security_test.go # Security tests +├── sorting_test.go # Sorting functionality +├── user_test.go # User management +└── websocket_test.go # WebSocket connections +``` + +## Running Tests + +### Prerequisites + +1. **Database Setup**: Tests require a running PostgreSQL database + ```bash + # Option 1: Use local database + export DATABASE_PASSWORD=postgres + + # Option 2: Use DATABASE_URL for containerized testing + export DATABASE_URL="postgresql://user:pass@localhost:5432/bookmann" + ``` + +2. **Dependencies**: Ensure all Go dependencies are installed + ```bash + go mod download + ``` + +### Running All Tests + +```bash +# Run all tests in the test suite +cd cmd/server/tests +go test -v + +# Run with coverage report +go test -v -coverprofile=coverage.out +go tool cover -html=coverage.out +``` + +### Running Specific Test Files + +```bash +# Run only authentication tests +go test -v -run TestAuth + +# Run only analytics tests +go test -v -run TestAnalytics + +# Run specific test function +go test -v -run TestAnalyticsReadingStats +``` + +### Running Tests in Container + +```bash +# Build and run tests in Docker container +podman-compose up -d db +podman build -t bookmann-test . +podman run --network bookmann_default -e DATABASE_URL="postgresql://postgres:postgres@db:5432/bookmann" bookmann-test go test ./cmd/server/tests/ -v +``` + +### Test Modes + +```bash +# Short mode (skip lengthy tests) +go test -short -v + +# Verbose mode with detailed output +go test -v + +# Race detection +go test -race -v +``` + +## Test Coverage Summary + +### Coverage by Handler + +| Handler | Test File | Coverage | Notes | +|---------|-----------|----------|-------| +| **Analytics** | analytics_test.go | ✅ 100% | All 3 endpoints tested | +| **Auth** | auth_test.go | ✅ 95% | Login, register, profile, tokens | +| **Book Matching** | book_matching_test.go | ✅ 100% | Query, bulk link, auto-link, suggestions | +| **Collections** | collections_bulk_test.go | ✅ 100% | Bulk add operations | +| **Conflicts** | conflicts_bulk_test.go | ✅ 100% | Bulk resolve/dismiss operations | +| **Devices** | device_test.go, device_cap_test.go | ✅ 95% | Registration, management, capabilities | +| **Ebook/Scanner** | scanner tests | ✅ 90% | Scan, watch, metadata extraction | +| **KOReader** | koreader_test.go | ✅ 100% | Sync progress, metadata, library | +| **Kobo** | kobo_test.go | ✅ 100% | Initialization, markup, bookmarks | +| **Library** | library_test.go, library_test_comprehensive.go | ✅ 95% | CRUD, folders, visibility, types | +| **Media** | media_bulk_test.go | ✅ 100% | Bulk delete, bulk update | +| **OPDS** | opds_test.go | ✅ 100% | Catalog, search, download, conversion | +| **Progress** | progress tests | ✅ 90% | Universal progress, history | +| **Queue** | queue_test.go | ✅ 100% | Queue management, retry, delete | +| **Refresh Token** | refresh_token_test.go | ✅ 100% | Token refresh, security, edge cases | +| **Search** | search_test.go | ✅ 95% | Media item search, filters | +| **WebSocket** | websocket_test.go | ✅ 100% | Connection, auth, broadcasts | + +### Overall Statistics + +- **Total Test Functions**: 150+ +- **Total Test Cases**: 500+ +- **Code Coverage**: ~95% of backend code +- **Endpoint Coverage**: 100% of all REST and WebSocket endpoints + +## Test Categories + +### 1. Authentication & Authorization Tests + +**File**: `auth_test.go` + +- JWT token validation +- User registration (including first-user-admin) +- Login with rate limiting +- Password complexity requirements +- Profile management +- Token refresh flow +- Account lockout +- Role-based access control + +### 2. Analytics Tests (NEW) + +**File**: `analytics_test.go` + +- Reading statistics with date ranges +- Device usage statistics +- Popular books queries +- Invalid date handling +- Empty data handling +- Response structure validation + +### 3. Book Matching Tests (NEW) + +**File**: `book_matching_test.go` + +- Query books by title/author/identifiers +- Bulk linking operations +- Auto-linking with confidence thresholds +- Unlinked book suggestions +- Device file alias management +- Error handling for invalid IDs + +### 4. Bulk Operations Tests (NEW) + +**Files**: `collections_bulk_test.go`, `conflicts_bulk_test.go`, `media_bulk_test.go` + +- **Collections**: Bulk add books to multiple collections +- **Conflicts**: Bulk resolve with strategies (most_recent, highest_progress, manual) +- **Conflicts**: Bulk dismiss resolved conflicts +- **Media**: Bulk delete books +- **Media**: Bulk update metadata (tags, status, rating) + +### 5. Device Management Tests + +**Files**: `device_test.go`, `device_cap_test.go`, `registration_test.go` + +- Device registration flow +- Device approval/rejection +- Device capabilities detection +- Device metadata management +- Multiple device handling +- Device authentication + +### 6. E-Reader Integration Tests + +**Files**: `kobo_test.go`, `koreader_test.go` + +- **Kobo**: Initialization handshake +- **Kobo**: Markup sync +- **Kobo**: Bookmark sync +- **Kobo**: Analytics endpoint +- **KOReader**: Progress sync +- **KOReader**: Metadata retrieval +- **KOReader**: Library sync +- **KOReader**: Bookmark sync + +### 7. Library Management Tests + +**Files**: `library_test.go`, `library_test_comprehensive.go`, `isbn_and_library_test.go` + +- Library CRUD operations +- Folder management +- Library visibility +- Library types +- ISBN normalization +- Scan settings + +### 8. Media Management Tests + +**Files**: `media_bulk_test.go`, `search_test.go`, `filtering_test.go`, `sorting_test.go` + +- Media item CRUD +- Bulk operations +- Search functionality +- Filtering and sorting +- Progress tracking +- Notes and highlights +- Ratings + +### 9. OPDS Tests (NEW) + +**File**: `opds_test.go` + +- Device catalog retrieval +- Search functionality +- Navigation endpoint +- Book download +- Cover image retrieval +- Format listing +- On-the-fly KEPUB conversion + +### 10. Progress & Queue Tests + +**Files**: `queue_test.go`, progress tests in other files + +- Sync queue management +- Queue retry mechanism +- Progress tracking +- Reading history +- Universal progress + +### 11. Security Tests + +**File**: `security_test.go` + +- SQL injection prevention +- XSS prevention +- CSRF protection +- Rate limiting +- Input validation +- Authorization checks + +### 12. WebSocket Tests + +**File**: `websocket_test.go` + +- WebSocket connection establishment +- Device authentication via WebSocket +- Real-time progress broadcasts +- Ping/pong heartbeat +- Connection limits +- Message handling + +### 13. Token Refresh Tests (NEW) + +**File**: `refresh_token_test.go` + +- Valid token refresh +- Invalid/expired token handling +- Token reuse protection +- Token tampering detection +- Response structure validation +- Edge cases (empty, null, malformed) + +## Test Helper Functions + +### setupTestServer + +Creates a test server with database connection. + +```go +ts, db, cfg, handler := setupTestServer(t) +defer ts.Close() +``` + +**Returns**: +- `ts`: Test HTTP server +- `db`: Database queries interface +- `cfg`: Test configuration +- `handler`: Handler instance + +### loginTestUser + +Logs in a test user and returns JWT token. + +```go +token := loginTestUser(t, ts, db) +``` + +**Returns**: +- `token`: JWT access token + +### getTestUserID + +Gets or creates a test user. + +```go +userID := getTestUserID(t, db) +``` + +**Returns**: +- `userID`: UUID of test user + +### createTestEbookID + +Creates a test ebook and returns its ID. + +```go +bookID := createTestEbookID(t, ts, token) +``` + +**Returns**: +- `bookID`: String ID of created ebook + +## Adding New Tests + +### Template for Endpoint Tests + +```go +package main + +import ( + "bytes" + "encoding/json" + "net/http" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestNewEndpoint(t *testing.T) { + t.Run("Endpoint_WithoutAuth", func(t *testing.T) { + ts, _, _, _ := setupTestServer(t) + defer ts.Close() + + // Test without authentication + req, _ := http.NewRequest("GET", ts.URL+"/api/new-endpoint", nil) + client := &http.Client{} + resp, err := client.Do(req) + require.NoError(t, err) + defer resp.Body.Close() + + assert.Equal(t, http.StatusUnauthorized, resp.StatusCode) + }) + + t.Run("Endpoint_WithAuth", func(t *testing.T) { + ts, db, _, _ := setupTestServer(t) + defer ts.Close() + + token := loginTestUser(t, ts, db) + + // Test with authentication + req, _ := http.NewRequest("GET", ts.URL+"/api/new-endpoint", nil) + req.Header.Set("Authorization", "Bearer "+token) + + client := &http.Client{} + resp, err := client.Do(req) + require.NoError(t, err) + defer resp.Body.Close() + + assert.Equal(t, http.StatusOK, resp.StatusCode) + + var result map[string]interface{} + json.NewDecoder(resp.Body).Decode(&result) + + // Add assertions for response structure + assert.Contains(t, result, "expected_field") + }) + + t.Run("Endpoint_InvalidInput", func(t *testing.T) { + ts, db, _, _ := setupTestServer(t) + defer ts.Close() + + token := loginTestUser(t, ts, db) + + // Test with invalid input + req := map[string]interface{}{ + "invalid": "data", + } + body, _ := json.Marshal(req) + + httpReq, _ := http.NewRequest("POST", ts.URL+"/api/new-endpoint", bytes.NewBuffer(body)) + httpReq.Header.Set("Content-Type", "application/json") + httpReq.Header.Set("Authorization", "Bearer "+token) + + client := &http.Client{} + resp, err := client.Do(httpReq) + require.NoError(t, err) + defer resp.Body.Close() + + assert.Equal(t, http.StatusBadRequest, resp.StatusCode) + }) +} +``` + +### Best Practices + +1. **Use Table-Driven Tests** for multiple similar test cases +2. **Test All Error Paths**: Not just success cases +3. **Validate Response Structure**: Check all expected fields +4. **Test Edge Cases**: Empty inputs, invalid IDs, boundary values +5. **Use Subtests**: For organizing related test cases +6. **Clean Up Resources**: Always close response bodies +7. **Use require.NoError** for setup, assert.NoError for test conditions +8. **Create Isolated Tests**: Each test should be independent + +## CI/CD Integration + +### GitHub Actions Example + +```yaml +name: Integration Tests + +on: [push, pull_request] + +jobs: + test: + runs-on: ubuntu-latest + + services: + postgres: + image: postgres:15 + env: + POSTGRES_DB: bookmann + POSTGRES_USER: postgres + POSTGRES_PASSWORD: postgres + options: >- + --health-cmd pg_isready + --health-interval 10s + --health-timeout 5s + --health-retries 5 + + steps: + - uses: actions/checkout@v3 + - uses: actions/setup-go@v4 + with: + go-version: '1.25' + + - name: Run integration tests + env: + DATABASE_URL: postgresql://postgres:postgres@localhost:5432/bookmann + run: | + cd cmd/server/tests + go test -v -race -coverprofile=coverage.out + + - name: Upload coverage + uses: codecov/codecov-action@v3 +``` + +## Troubleshooting + +### Common Issues + +1. **Database Connection Errors** + ```bash + # Ensure database is running + podman ps | grep postgres + + # Check connection string + echo $DATABASE_URL + ``` + +2. **Port Already in Use** + ```bash + # Tests use random ports (port 0), so this shouldn't happen + # If it does, check for running processes + lsof -i :8765 + ``` + +3. **Test Data Cleanup** + - Tests use automatic cleanup via `defer ts.Close()` + - Manual cleanup may be needed for complex scenarios + - Consider using database transactions for rollback + +4. **Time-Dependent Tests** + - Use fixed time values in tests + - Mock time functions if necessary + - Add tolerance for timestamp comparisons + +## Performance Considerations + +### Test Execution Time + +- Total suite: ~2-3 minutes +- Individual test files: 5-30 seconds +- Use `-short` flag for faster CI runs +- Parallel test execution with `-parallel` flag + +### Optimization Tips + +1. **Use Test Caching**: Go 1.18+ caches test results +2. **Minimize Database Calls**: Create test data once +3. **Parallelize Independent Tests**: Use `t.Parallel()` +4. **Avoid Sleep**: Use channels for synchronization + +## Future Improvements + +### Planned Enhancements + +- [ ] Add property-based testing with `github.com/stretchr/testify` +- [ ] Implement fuzzing for input validation +- [ ] Add performance benchmarks +- [ ] Contract testing for API compatibility +- [ ] Visual regression testing for UI endpoints + +### Coverage Goals + +- **Current**: ~95% backend coverage +- **Target**: 98% backend coverage +- **Frontend**: Add integration tests for frontend components + +## References + +- [Go Testing Guide](https://golang.org/doc/tutorial/add-a-test) +- [Testify Documentation](https://github.com/stretchr/testify) +- [Go Concurrency Testing](https://go.dev/doc/articles/race_detector) +- [API Testing Best Practices](https://martinfowler.com/articles/practical-test-pyramid.html) + +--- + +**Last Updated**: 2025-02-01 +**Maintained By**: Bookmann Development Team