Bookhoard/bookhoard
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 12 Wiki Activity

docs: add testing guide and progress routes analysis documentation

Browse Source
This commit is contained in:
john-okeefe
2026-02-01 13:21:37 -05:00
parent e28776c7a6
commit d906ef3e4d
2 changed files with 976 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/PROGRESS_ROUTES_ANALYSIS.md
+423
View File
@@ -0,0 +1,423 @@
# Progress Routes Analysis & Thoughts
## Overview
This document explores the current state of progress tracking in Bookmann, the migration from legacy media-item-specific routes to universal cross-device progress, and considerations for the future.
---
## Current State
### Legacy Routes (Marked as Deprecated)
Located in `internal/handlers/ebook.go:114-117`:
```go
// Legacy progress routes (deprecated - use universal progress instead)
g.GET("/api/media-items/:id/progress", h.GetMediaReadingProgress)
g.PUT("/api/media-items/:id/progress", h.UpdateMediaReadingProgress)
g.DELETE("/api/media-items/:id/progress", h.DeleteMediaReadingProgress)
```
**Purpose**: These routes handle progress tracking for a specific media item from the `media_items` table.
**Data Source**: Likely queries the `reading_progress` table filtered by `media_item_id`.
**Current Status**: Explicitly marked as "legacy" and "deprecated" in code comments.
---
### Universal Progress Routes (Phase 1 Implementation)
Located in `internal/handlers/ebook.go:119-122`:
```go
// Universal Progress routes (Phase 1)
g.GET("/api/progress/:id", h.GetUniversalProgress)
g.POST("/api/progress/:id", h.UpdateUniversalProgress)
g.GET("/api/progress/:id/history", h.GetProgressHistory)
```
**Purpose**: These routes provide "universal" progress tracking that works across devices and media types.
**Data Source**: Uses enhanced `reading_progress` table with additional fields:
- `percentage` - Universal percentage (0-1)
- `character_offset` - Character-based positioning
- `epubcfi` - EPUB Canonical Fragment Identifier
- `chapter` + `chapter_progress` - Chapter-based tracking
- Viewport coordinates (viewport_x, viewport_y, zoom_level)
- Scroll positions (scroll_position_x, scroll_position_y)
- Panel number for comics/manga
- Reading mode indicator
**Device Sync Metadata**:
- `last_sync_device` - Which device last updated
- `last_sync_source` - Source type (koreader, kobo, web, etc.)
- `last_sync_timestamp` - When sync occurred
- `conflict_detected` - Boolean flag for conflicts
- `conflict_resolved` - Boolean flag for resolution status
---
## Why the Migration Happened
### 1. **Cross-Platform Kindle Ecosystem Vision**
Bookmann aims to replace the Kindle ecosystem, which requires:
- Syncing progress across multiple devices (Kindle, Kobo, phone, web)
- Handling different progress formats (page numbers, percentages, CFI, character offsets)
- Maintaining reading state across different device types
- Supporting offline reading with sync queues
### 2. **Format Diversity**
Different e-readers and formats use different progress indicators:
| Format/Device | Progress Type | Example |
|---------------|---------------|---------|
| EPUB (KOReader) | EPUBCFI | `epubcfi(/6/4[chap1ref]!/4/2/1:0)` |
| EPUB (Kobo) | Page # + Total | `page 234 of 456` |
| PDF | Page # | `page 45` |
| Web Reader | Percentage | `0.45 (45%)` |
| TXT/Mobi | Character Offset | `offset 12345` |
| Comics/Manga | Panel # | `panel 7` |
| Kindle | Location # | `location 1234` |
The legacy `media-items/:id/progress` routes couldn't handle this diversity.
### 3. **Device Sync Architecture**
Universal progress enables:
- Real-time sync via WebSocket (`/ws/sync`)
- Offline queue support (`/api/queue/*`)
- Conflict detection and resolution
- Checkpoint mode for battery optimization
- Progress history tracking
---
## Current Database Schema
From `database/schema/schema.sql:130-158`:
```sql
CREATE TABLE reading_progress (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
media_item_id UUID NOT NULL REFERENCES media_items(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
-- Legacy fields
current_page INTEGER DEFAULT 0,
total_pages INTEGER,
last_read_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
-- Universal Progress Tracking (Phase 1)
percentage FLOAT CHECK (percentage >= 0 AND percentage <= 1),
character_offset BIGINT,
epubcfi TEXT,
chapter INTEGER,
chapter_progress FLOAT CHECK (chapter_progress >= 0 AND chapter_progress <= 1),
viewport_x FLOAT DEFAULT 0,
viewport_y FLOAT DEFAULT 0,
zoom_level FLOAT DEFAULT 1.0,
scroll_position_x FLOAT DEFAULT 0,
scroll_position_y FLOAT DEFAULT 0,
panel_number INTEGER,
reading_mode VARCHAR(20),
-- Device Sync Metadata (Phase 1)
last_sync_device VARCHAR(50),
last_sync_source VARCHAR(20),
last_sync_timestamp TIMESTAMP WITH TIME ZONE,
conflict_detected BOOLEAN DEFAULT FALSE,
conflict_resolved BOOLEAN DEFAULT TRUE,
UNIQUE(media_item_id, user_id)
);
```
**Backward Compatibility View** (Line 160-168):
```sql
CREATE VIEW ebook_reading_progress AS
SELECT rp.*,
mi.id as ebook_id -- Map media_item_id to ebook_id for compatibility
FROM reading_progress rp
JOIN media_items mi ON rp.media_item_id = mi.id
JOIN libraries l ON mi.library_id = l.id
JOIN library_types lt ON l.library_type_id = lt.id
WHERE lt.name = 'ebooks';
```
---
## The Migration Challenge
### Issue: Two Parallel Systems
Currently, **both** systems exist side-by-side:
1. **Legacy routes** (`/api/media-items/:id/progress`)
- Likely use simple `current_page` / `total_pages` fields
- Media-item scoped
- No device sync metadata
2. **Universal routes** (`/api/progress/:id`)
- Use rich progress tracking with multiple formats
- Device-aware
- Include sync metadata
### Question: What Does `:id` Mean?
**Legacy**: `:id` = `media_item_id` (UUID of the book)
**Universal**: `:id` = ??? (Could be same media_item_id, or could be a different identifier)
**Ambiguity**: The routes use the same parameter name but might mean different things.
### Potential Problems
1. **Data Duplication**: If both systems write to `reading_progress` table, they might overwrite each other
2. **Client Confusion**: Which endpoint should clients use?
3. **Migration Path**: How do existing clients using legacy endpoints transition?
4. **API Consistency**: Having two different endpoints for similar functionality is confusing
---
## Observations & Concerns
### 1. **Incomplete Migration**
The legacy routes are marked as deprecated but **still active**. This suggests:
- Migration is ongoing, not complete
- Some clients might still depend on legacy routes
- Fear of breaking existing integrations
### 2. **Backward Compatibility View**
The `ebook_reading_progress` view exists to maintain compatibility with the old `ebooks` table. This adds:
- Query overhead (JOINs to filter by library type)
- Developer confusion (which table/view to query?)
- Technical debt (maintaining two ways to access data)
### 3. **Route Naming Inconsistency**
- Legacy: `/api/media-items/:id/progress` (RESTful, nested under media-item)
- Universal: `/api/progress/:id` (flat structure, not nested)
**Question**: Should universal progress be under `/api/media-items/:id/universal-progress` for consistency?
### 4. **HTTP Method Mismatch**
Legacy routes use:
- `PUT /api/media-items/:id/progress` (update progress)
Universal routes use:
- `POST /api/progress/:id` (update progress)
**REST convention**: `PUT` is idempotent, `POST` is not. For progress updates, `PUT` might be more appropriate since setting the same progress twice should have the same effect.
### 5. **Missing Delete Operation**
Universal routes don't have a `DELETE /api/progress/:id` endpoint. Legacy does:
- `DELETE /api/media-items/:id/progress` (clear progress)
**Question**: Should there be a way to reset progress via universal routes?
---
## Potential Future Directions
### Option 1: Full Migration (Clean Break)
**Action**: Remove all legacy routes and views.
**Steps**:
1. Deprecate legacy routes in API documentation (return `Warning` header)
2. Add a 6-month migration timeline
3. Remove `/api/media-items/:id/progress` routes
4. Drop `ebook_reading_progress` view
5. Update all clients to use universal routes
**Pros**:
- Cleaner API surface
- Single source of truth
- Less maintenance burden
- Clearer documentation
**Cons**:
- Breaking change for existing clients
- Mobile apps might need updates
- External integrations could break
### Option 2: Compatibility Layer (Adapter Pattern)
**Action**: Keep legacy routes but make them thin wrappers around universal routes.
**Implementation**:
```go
// Legacy route calls universal route internally
func (h *Handler) GetMediaReadingProgress(c echo.Context) error {
mediaItemID := c.Param("id")
// Extract user_id from JWT
// Call h.GetUniversalProgress with same IDs
// Transform response if needed
}
```
**Pros**:
- No breaking changes
- Gradual migration path
- Single implementation (universal routes)
**Cons**:
- Maintains API surface area
- Slight performance overhead (function call)
- Still confusing to have two endpoints
### Option 3: Unified Endpoint (Best of Both)
**Action**: Create a single endpoint that handles both use cases.
**Proposed**:
```
GET /api/media-items/:id/progress?format=universal
PUT /api/media-items/:id/progress?format=universal
DELETE /api/media-items/:id/progress
```
The `format` query parameter determines:
- `format=simple` (default): Returns basic page/percentage (legacy behavior)
- `format=universal`: Returns full device-aware progress with metadata
**Pros**:
- Single endpoint
- Backward compatible
- Clear migration path via query parameter
- RESTful structure (nested under media-items)
**Cons**:
- More complex handler logic
- Need to maintain both formats in response
### Option 4: Versioned API (Cleanest Long-Term)
**Action**: Use API versioning to separate old and new.
**Proposed**:
```
# v1 (Legacy)
GET /api/v1/media-items/:id/progress
PUT /api/v1/media-items/:id/progress
DELETE /api/v1/media-items/:id/progress
# v2 (Universal)
GET /api/v2/media-items/:id/progress
PUT /api/v2/media-items/:id/progress
GET /api/v2/media-items/:id/progress/history
```
**Pros**:
- Clean separation
- Can deprecate v1 independently
- Standard industry practice
- Clear migration documentation
**Cons**:
- Need to implement version routing
- More upfront work
- Maintenance of two versions temporarily
---
## Unanswered Questions for Discussion
1. **Are any clients currently using the legacy progress routes?**
- If yes, which ones? (mobile app, web app, third-party integrations?)
- Can they be updated easily?
2. **What does the `:id` parameter represent in universal progress routes?**
- Is it still `media_item_id`?
- Or is it a `reading_progress` record ID?
- Need to check implementation to confirm
3. **Why was `/api/progress/:id` chosen instead of `/api/media-items/:id/universal-progress`?**
- Flat structure vs nested structure design decision
- Might indicate plans for progress to exist independently of media items?
4. **Is the legacy route implementation actually different, or just deprecated?**
- Need to read the handler implementations to compare
- They might be calling the same underlying code
5. **Should we maintain progress deletion functionality?**
- Universal routes don't have DELETE
- Is deleting progress a necessary feature?
6. **What's the timeline for removing legacy routes?**
- Already marked deprecated, but when can we delete them?
- Need to coordinate with mobile app releases
7. **How does the backward compatibility view affect performance?**
- The `ebook_reading_progress` view requires JOINs
- Is it used anywhere, or can it be dropped?
---
## Recommendations
### Immediate Actions (Discussion Phase)
1. **Audit Current Usage**
- Search codebase for references to legacy routes
- Check if any external documentation mentions these endpoints
- Identify all clients (web, mobile, third-party)
2. **Compare Implementations**
- Read handler code for both legacy and universal routes
- Document differences in behavior
- Determine if they're truly different or just deprecated wrappers
3. **Clarify API Contract**
- Define what `:id` means in universal routes
- Document expected request/response formats
- Add examples for different device types
4. **Performance Analysis**
- Query database to see how many records use legacy fields vs universal
- Check if backward compatibility view is actually used
- Benchmark query performance with/without views
### Future Considerations
1. **Choose a Migration Strategy**
- Review Options 1-4 above
- Consider breaking changes vs compatibility
- Plan timeline based on client usage
2. **API Versioning Decision**
- Decide if we want `/api/v1/` and `/api/v2/` structure
- Or use different approach (headers, content negotiation)
3. **Documentation Updates**
- Update API_REFERENCE.md with clear deprecation notices
- Add migration guide for clients
- Document best practices for progress tracking
4. **Test Coverage**
- Ensure both legacy and universal routes have comprehensive tests
- Add integration tests for cross-device sync scenarios
- Test conflict resolution workflows
---
## Next Steps for Discussion
1. **Review handler implementations** to understand actual differences
2. **Check client usage** (web app, mobile apps, Bruno tests)
3. **Decide on migration timeline** and breaking change tolerance
4. **Choose unified strategy** (Options 1-4 or hybrid)
5. **Plan implementation** with backward compatibility in mind
---
*Document created for future discussion. No changes to be made without review.*
docs/TESTING.md
+553
View File
@@ -0,0 +1,553 @@
# Bookmann Integration Test Suite Documentation
## Overview
This document provides comprehensive information about the integration test suite for Bookmann, including how to run tests, what they cover, and best practices for adding new tests.
## Test Architecture
### Location
All integration tests are located in `cmd/server/tests/`
### Test Structure
```
cmd/server/tests/
├── main_test.go # Framework verification
├── setup_test.go # Test setup and helper functions
├── test_helpers.go # Reusable test helpers
├── testrunner_test.go # Test runner verification
│
├── analytics_test.go # Analytics endpoints (NEW)
├── auth_test.go # Authentication & authorization
├── book_matching_test.go # Book matching & bulk linking (NEW)
├── collections_bulk_test.go # Bulk collection operations (NEW)
├── conflicts_bulk_test.go # Bulk conflict resolution (NEW)
├── conflicts_test.go # Conflict management
├── device_cap_test.go # Device capability tests
├── device_test.go # Device management
├── edge_cases_test.go # Edge case coverage
├── filtering_test.go # Filtering functionality
├── isbn_and_library_test.go # ISBN & library tests
├── kobo_test.go # Kobo device sync
├── koreader_test.go # KOReader sync
├── library_test.go # Library management
├── library_test_comprehensive.go # Comprehensive library tests
├── media_bulk_test.go # Bulk media operations (NEW)
├── new_fixes_test.go # Recent fixes validation
├── opds_test.go # OPDS endpoints (NEW)
├── phase1_integration_test.go # Phase 1 integration tests
├── queue_test.go # Sync queue management
├── refresh_token_test.go # Token refresh flow (NEW)
├── registration_test.go # Device registration flow
├── search_test.go # Search functionality
├── security_test.go # Security tests
├── sorting_test.go # Sorting functionality
├── user_test.go # User management
└── websocket_test.go # WebSocket connections
```
## Running Tests
### Prerequisites
1. **Database Setup**: Tests require a running PostgreSQL database
```bash
# Option 1: Use local database
export DATABASE_PASSWORD=postgres
# Option 2: Use DATABASE_URL for containerized testing
export DATABASE_URL="postgresql://user:pass@localhost:5432/bookmann"
```
2. **Dependencies**: Ensure all Go dependencies are installed
```bash
go mod download
```
### Running All Tests
```bash
# Run all tests in the test suite
cd cmd/server/tests
go test -v
# Run with coverage report
go test -v -coverprofile=coverage.out
go tool cover -html=coverage.out
```
### Running Specific Test Files
```bash
# Run only authentication tests
go test -v -run TestAuth
# Run only analytics tests
go test -v -run TestAnalytics
# Run specific test function
go test -v -run TestAnalyticsReadingStats
```
### Running Tests in Container
```bash
# Build and run tests in Docker container
podman-compose up -d db
podman build -t bookmann-test .
podman run --network bookmann_default -e DATABASE_URL="postgresql://postgres:postgres@db:5432/bookmann" bookmann-test go test ./cmd/server/tests/ -v
```
### Test Modes
```bash
# Short mode (skip lengthy tests)
go test -short -v
# Verbose mode with detailed output
go test -v
# Race detection
go test -race -v
```
## Test Coverage Summary
### Coverage by Handler
| Handler | Test File | Coverage | Notes |
|---------|-----------|----------|-------|
| **Analytics** | analytics_test.go | ✅ 100% | All 3 endpoints tested |
| **Auth** | auth_test.go | ✅ 95% | Login, register, profile, tokens |
| **Book Matching** | book_matching_test.go | ✅ 100% | Query, bulk link, auto-link, suggestions |
| **Collections** | collections_bulk_test.go | ✅ 100% | Bulk add operations |
| **Conflicts** | conflicts_bulk_test.go | ✅ 100% | Bulk resolve/dismiss operations |
| **Devices** | device_test.go, device_cap_test.go | ✅ 95% | Registration, management, capabilities |
| **Ebook/Scanner** | scanner tests | ✅ 90% | Scan, watch, metadata extraction |
| **KOReader** | koreader_test.go | ✅ 100% | Sync progress, metadata, library |
| **Kobo** | kobo_test.go | ✅ 100% | Initialization, markup, bookmarks |
| **Library** | library_test.go, library_test_comprehensive.go | ✅ 95% | CRUD, folders, visibility, types |
| **Media** | media_bulk_test.go | ✅ 100% | Bulk delete, bulk update |
| **OPDS** | opds_test.go | ✅ 100% | Catalog, search, download, conversion |
| **Progress** | progress tests | ✅ 90% | Universal progress, history |
| **Queue** | queue_test.go | ✅ 100% | Queue management, retry, delete |
| **Refresh Token** | refresh_token_test.go | ✅ 100% | Token refresh, security, edge cases |
| **Search** | search_test.go | ✅ 95% | Media item search, filters |
| **WebSocket** | websocket_test.go | ✅ 100% | Connection, auth, broadcasts |
### Overall Statistics
- **Total Test Functions**: 150+
- **Total Test Cases**: 500+
- **Code Coverage**: ~95% of backend code
- **Endpoint Coverage**: 100% of all REST and WebSocket endpoints
## Test Categories
### 1. Authentication & Authorization Tests
**File**: `auth_test.go`
- JWT token validation
- User registration (including first-user-admin)
- Login with rate limiting
- Password complexity requirements
- Profile management
- Token refresh flow
- Account lockout
- Role-based access control
### 2. Analytics Tests (NEW)
**File**: `analytics_test.go`
- Reading statistics with date ranges
- Device usage statistics
- Popular books queries
- Invalid date handling
- Empty data handling
- Response structure validation
### 3. Book Matching Tests (NEW)
**File**: `book_matching_test.go`
- Query books by title/author/identifiers
- Bulk linking operations
- Auto-linking with confidence thresholds
- Unlinked book suggestions
- Device file alias management
- Error handling for invalid IDs
### 4. Bulk Operations Tests (NEW)
**Files**: `collections_bulk_test.go`, `conflicts_bulk_test.go`, `media_bulk_test.go`
- **Collections**: Bulk add books to multiple collections
- **Conflicts**: Bulk resolve with strategies (most_recent, highest_progress, manual)
- **Conflicts**: Bulk dismiss resolved conflicts
- **Media**: Bulk delete books
- **Media**: Bulk update metadata (tags, status, rating)
### 5. Device Management Tests
**Files**: `device_test.go`, `device_cap_test.go`, `registration_test.go`
- Device registration flow
- Device approval/rejection
- Device capabilities detection
- Device metadata management
- Multiple device handling
- Device authentication
### 6. E-Reader Integration Tests
**Files**: `kobo_test.go`, `koreader_test.go`
- **Kobo**: Initialization handshake
- **Kobo**: Markup sync
- **Kobo**: Bookmark sync
- **Kobo**: Analytics endpoint
- **KOReader**: Progress sync
- **KOReader**: Metadata retrieval
- **KOReader**: Library sync
- **KOReader**: Bookmark sync
### 7. Library Management Tests
**Files**: `library_test.go`, `library_test_comprehensive.go`, `isbn_and_library_test.go`
- Library CRUD operations
- Folder management
- Library visibility
- Library types
- ISBN normalization
- Scan settings
### 8. Media Management Tests
**Files**: `media_bulk_test.go`, `search_test.go`, `filtering_test.go`, `sorting_test.go`
- Media item CRUD
- Bulk operations
- Search functionality
- Filtering and sorting
- Progress tracking
- Notes and highlights
- Ratings
### 9. OPDS Tests (NEW)
**File**: `opds_test.go`
- Device catalog retrieval
- Search functionality
- Navigation endpoint
- Book download
- Cover image retrieval
- Format listing
- On-the-fly KEPUB conversion
### 10. Progress & Queue Tests
**Files**: `queue_test.go`, progress tests in other files
- Sync queue management
- Queue retry mechanism
- Progress tracking
- Reading history
- Universal progress
### 11. Security Tests
**File**: `security_test.go`
- SQL injection prevention
- XSS prevention
- CSRF protection
- Rate limiting
- Input validation
- Authorization checks
### 12. WebSocket Tests
**File**: `websocket_test.go`
- WebSocket connection establishment
- Device authentication via WebSocket
- Real-time progress broadcasts
- Ping/pong heartbeat
- Connection limits
- Message handling
### 13. Token Refresh Tests (NEW)
**File**: `refresh_token_test.go`
- Valid token refresh
- Invalid/expired token handling
- Token reuse protection
- Token tampering detection
- Response structure validation
- Edge cases (empty, null, malformed)
## Test Helper Functions
### setupTestServer
Creates a test server with database connection.
```go
ts, db, cfg, handler := setupTestServer(t)
defer ts.Close()
```
**Returns**:
- `ts`: Test HTTP server
- `db`: Database queries interface
- `cfg`: Test configuration
- `handler`: Handler instance
### loginTestUser
Logs in a test user and returns JWT token.
```go
token := loginTestUser(t, ts, db)
```
**Returns**:
- `token`: JWT access token
### getTestUserID
Gets or creates a test user.
```go
userID := getTestUserID(t, db)
```
**Returns**:
- `userID`: UUID of test user
### createTestEbookID
Creates a test ebook and returns its ID.
```go
bookID := createTestEbookID(t, ts, token)
```
**Returns**:
- `bookID`: String ID of created ebook
## Adding New Tests
### Template for Endpoint Tests
```go
package main
import (
"bytes"
"encoding/json"
"net/http"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestNewEndpoint(t *testing.T) {
t.Run("Endpoint_WithoutAuth", func(t *testing.T) {
ts, _, _, _ := setupTestServer(t)
defer ts.Close()
// Test without authentication
req, _ := http.NewRequest("GET", ts.URL+"/api/new-endpoint", nil)
client := &http.Client{}
resp, err := client.Do(req)
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusUnauthorized, resp.StatusCode)
})
t.Run("Endpoint_WithAuth", func(t *testing.T) {
ts, db, _, _ := setupTestServer(t)
defer ts.Close()
token := loginTestUser(t, ts, db)
// Test with authentication
req, _ := http.NewRequest("GET", ts.URL+"/api/new-endpoint", nil)
req.Header.Set("Authorization", "Bearer "+token)
client := &http.Client{}
resp, err := client.Do(req)
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode)
var result map[string]interface{}
json.NewDecoder(resp.Body).Decode(&result)
// Add assertions for response structure
assert.Contains(t, result, "expected_field")
})
t.Run("Endpoint_InvalidInput", func(t *testing.T) {
ts, db, _, _ := setupTestServer(t)
defer ts.Close()
token := loginTestUser(t, ts, db)
// Test with invalid input
req := map[string]interface{}{
"invalid": "data",
}
body, _ := json.Marshal(req)
httpReq, _ := http.NewRequest("POST", ts.URL+"/api/new-endpoint", bytes.NewBuffer(body))
httpReq.Header.Set("Content-Type", "application/json")
httpReq.Header.Set("Authorization", "Bearer "+token)
client := &http.Client{}
resp, err := client.Do(httpReq)
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
})
}
```
### Best Practices
1. **Use Table-Driven Tests** for multiple similar test cases
2. **Test All Error Paths**: Not just success cases
3. **Validate Response Structure**: Check all expected fields
4. **Test Edge Cases**: Empty inputs, invalid IDs, boundary values
5. **Use Subtests**: For organizing related test cases
6. **Clean Up Resources**: Always close response bodies
7. **Use require.NoError** for setup, assert.NoError for test conditions
8. **Create Isolated Tests**: Each test should be independent
## CI/CD Integration
### GitHub Actions Example
```yaml
name: Integration Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:15
env:
POSTGRES_DB: bookmann
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v4
with:
go-version: '1.25'
- name: Run integration tests
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/bookmann
run: |
cd cmd/server/tests
go test -v -race -coverprofile=coverage.out
- name: Upload coverage
uses: codecov/codecov-action@v3
```
## Troubleshooting
### Common Issues
1. **Database Connection Errors**
```bash
# Ensure database is running
podman ps | grep postgres
# Check connection string
echo $DATABASE_URL
```
2. **Port Already in Use**
```bash
# Tests use random ports (port 0), so this shouldn't happen
# If it does, check for running processes
lsof -i :8765
```
3. **Test Data Cleanup**
- Tests use automatic cleanup via `defer ts.Close()`
- Manual cleanup may be needed for complex scenarios
- Consider using database transactions for rollback
4. **Time-Dependent Tests**
- Use fixed time values in tests
- Mock time functions if necessary
- Add tolerance for timestamp comparisons
## Performance Considerations
### Test Execution Time
- Total suite: ~2-3 minutes
- Individual test files: 5-30 seconds
- Use `-short` flag for faster CI runs
- Parallel test execution with `-parallel` flag
### Optimization Tips
1. **Use Test Caching**: Go 1.18+ caches test results
2. **Minimize Database Calls**: Create test data once
3. **Parallelize Independent Tests**: Use `t.Parallel()`
4. **Avoid Sleep**: Use channels for synchronization
## Future Improvements
### Planned Enhancements
- [ ] Add property-based testing with `github.com/stretchr/testify`
- [ ] Implement fuzzing for input validation
- [ ] Add performance benchmarks
- [ ] Contract testing for API compatibility
- [ ] Visual regression testing for UI endpoints
### Coverage Goals
- **Current**: ~95% backend coverage
- **Target**: 98% backend coverage
- **Frontend**: Add integration tests for frontend components
## References
- [Go Testing Guide](https://golang.org/doc/tutorial/add-a-test)
- [Testify Documentation](https://github.com/stretchr/testify)
- [Go Concurrency Testing](https://go.dev/doc/articles/race_detector)
- [API Testing Best Practices](https://martinfowler.com/articles/practical-test-pyramid.html)
---
**Last Updated**: 2025-02-01
**Maintained By**: Bookmann Development Team