Fix database connection exhaustion in tests by setting max_conns=1
when creating pgxpool via pgxpool.ParseConfig().
- Update setupTestServer() in test_helpers.go
- Update setupSyncTestDB() in sync_integration_test.go
This reduces per-test connection usage from 4 to 1, keeping total
connections well under PostgreSQL's default max_connections=100.
78 tests × 1 connection = 78 connections (down from 312 potential)
Fixes test failures: "FATAL: sorry, too many clients already"
See PROJECT_GUIDELINES.md Testing section for details.
Test expectations in TestMediaItemISBNNormalization were incorrect:
- Tests were expecting 12-digit outputs for 13-digit inputs
- Updated to expect correct 13-digit normalized outputs
This fixes the failing normalization tests.
Add addFolderToLibrary helper function:
- Creates folder via POST /api/libraries/{id}/folders
- Called after each createTestLibrary in test files
This fixes failing tests where media item creation failed with:
'Cannot add media items to a library with no folders.'
Tests now properly create libraries with folders before adding media items.
Enhance NormalizeISBN to validate and convert ISBNs:
- Validate length (10 or 13 digits), return error if invalid
- Convert ISBN-10 to ISBN-13 by prefixing '978' and recalculating checksum
- Add NormalizeISBNSafe for backward compatibility in scanners
This ensures all ISBNs stored in database are valid ISBN-13 format.
Update test assertions in TestCollectionsBulkOperations to expect
'added' instead of 'success' in the response, matching the handler
change made in the bulk operations rename.
Fixes:
- BulkAddBooks_InvalidCollectionID: assert 'added' field exists
- BulkAddBooks_SingleOperation: assert 'added' field exists
Add strict validation to return 400 Bad Request when any media_item_id
is empty in the bulk-update request, rather than treating it as a
partial failure with 200 OK.
This aligns the handler behavior with test expectations for the
BulkUpdateBooks_EmptyBookIDs test case.
- Update api-reference.md with new endpoint paths
- Update api-reference.md Books API section → Media Items API section
- Update index.md Books API link → Media Items API
- Update get_shelf.md cover_url reference from /api/books/ to /api/media-items/
- Add comprehensive documentation for bulk delete endpoint
- Add comprehensive documentation for bulk update endpoint
- Add comprehensive documentation for download endpoint
- Document all request/response fields with correct names
- Include examples and error codes
- Add notes on tag normalization and partial success
- Update all test URLs from /api/books/ to /api/media-items/
- Update request structures: book_ids → media_item_ids
- Update bulk-update request format to array of operations
- Update response assertions: success → deleted/updated
- Update result assertions: book_id → media_item_id
The test was sending a single object with a "meta" field, but the handler
expects an array of KoboAnalyticsTest objects (matching other Kobo endpoints).
Changed:
- Removed unsupported "meta" field
- Converted single object to array format
- Now matches Kobo protocol pattern used by /markup and /bookmark endpoints
All Kobo tests now pass:
- TestKoboInitialization ✅
- TestKoboLibrarySync ✅
- TestKoboMarkupSync ✅
- TestKoboBookmarkSync ✅
- TestKoboAnalyticsGettests ✅
- TestKoboDeviceHeaderParsing ✅
The TestKoboInitialization test was calling getTestUserID() explicitly
on line 25, but loginTestUser() already calls this internally. This caused
the test user to be deleted and recreated after login, leading to
inconsistent state and HTTP 500 errors when creating libraries.
After removing the redundant call:
- TestKoboInitialization now passes
- All Kobo sync tests pass successfully
- Add DROP TRIGGER IF EXISTS before CREATE TRIGGER
- Fixes 'trigger already exists' error during schema initialization
- Allows schema to run multiple times safely
- Try multiple locations for schema.sql file
- Support both local dev and containerized deployment paths
- Add informative logging when schema is loaded
- Prevent runtime.Caller issues in containers
Locations checked:
- database/schema/schema.sql (working directory)
- /app/database/schema/schema.sql (container)
- ../database/schema/schema.sql (relative)
- ../../database/schema/schema.sql (relative)
This fixes the 'no such file or directory' error in production containers.
- Add schema initialization call after database connection
- Initialize schema before handler creation
- Fatal on failure (schema is critical for app to function)
- Clear log messages show initialization progress
Server startup flow:
1. Load config
2. Connect to database
3. Initialize schema (NEW - ensures all tables/functions exist)
4. Create handlers and services
5. Start server
- Create internal/database/schema.go with full initialization logic
- Parse table names from schema.sql using regex (handles both formats)
- Execute schema in atomic transaction
- Verify all expected tables exist
- Verify all critical functions exist (6 functions)
- PostgreSQL advisory locking with 30-second timeout
- Self-healing from partial/corrupted state
- Load schema.sql from filesystem at runtime
Features:
- Defensive regex handles IF NOT EXISTS and legacy CREATE TABLE
- Lock timeout prevents indefinite hangs
- Function verification ensures sync operations work
- Clear error messages with debug hints
- Convert 18 CREATE TABLE → CREATE TABLE IF NOT EXISTS (27 total)
- Convert 62 CREATE INDEX → CREATE INDEX IF NOT EXISTS (82 total)
- Add ON CONFLICT to 2 INSERT statements (3 total)
- Verify 8 ALTER TABLE already have IF NOT EXISTS
- Verify 6 CREATE FUNCTION use OR REPLACE
Schema is now fully idempotent and safe for automatic initialization on every startup.
- Fix regex pattern to handle both IF NOT EXISTS and legacy CREATE TABLE formats
- Add 30-second lock timeout to prevent indefinite hangs
- Add function verification (6 critical functions checked)
- Document 8 ALTER TABLE statements already idempotent
- Document 6 CREATE FUNCTION statements use OR REPLACE
- Add time import for timeout support
- Update verification checklist with new requirements
- Update log messages to show table and function counts
The first GetReadingStats_WithAuth_DefaultDates test was expecting
'total_books' and 'total_reading_time' fields that don't exist in the
API response. The second duplicate test correctly expects
'total_books_read' and 'total_reading_time_minutes'.
This resolves the TestAnalyticsReadingStats failure.
Finish migrating all test files to the new TestServerSetup pattern
introduced by the goroutine cleanup refactoring. This resolves all
remaining compilation errors in the test suite.
Changes:
- device_cap_test.go: Fix undefined ts references (7 instances)
* Replace ts.URL with setup.Server.URL in all test functions
* Fix URL references in t.Run subtest closures
- queue_test.go: Fix undefined db and helper function issues (5 instances)
* Replace db.CreateDevice with setup.DB.CreateDevice
* Fix loginAdminUser() to use ts/db parameters instead of setup
* Fix loginUserWithID() to use ts parameter instead of setup
- websocket_test.go: Convert 5 tests to new TestServerSetup pattern
* Replace old pattern (ts, queries, _) with new pattern (setup)
* Update all resource references to use setup.Server and setup.DB
* Fix getTestUserID calls to include t parameter
Build Impact:
- All compilation errors resolved
- Integration tests now compile successfully
- No functional changes to test logic
Related: TestServerSetup cleanup pattern (TEST_CLEANUP_PATTERN.md)
BREAKING CHANGE: setupTestServer() now returns *TestServerSetup instead of (*httptest.Server, *database.Queries, *config.Config)
This fixes the database connection and goroutine leak issues where:
- Each test created a new pgxpool (default max_conns = 4)
- connManager.StartCleanupTask() goroutine was never stopped
- queueProcessor.Start() goroutine was never stopped
- ~160 tests = potential 640+ leaked connections
New TestServerSetup struct provides:
- Automatic cleanup via t.Cleanup()
- Proper goroutine cancellation
- Database pool closing
- Thread-safe close() method with mutex
Phase 1 of test cleanup refactor.
- Fix TestKoboInitialization: use setupDeviceTest() for device creation
- Fix TestKoboLibrarySync: remove /test-token/ route path, add device auth
- Fix TestKoboMarkupSync: add device auth and last-read-place test case
- Fix TestKoboBookmarkSync: add device auth and last-read-place test case
- Fix TestKoboAnalyticsGettests: add device authentication
- Add debug logging to all test functions
- Remove unused imports (config, database, middleware, router, services, sync)
Phase 3 of KOBO_IMPLEMENTATION_PLAN.md completed (Steps 7-12).
All tests now use proper device authentication (Bearer tokens + x-kobo-device headers)
and include test cases for the new last-read-place bookmark feature.
- Add nil UUID checks after mapContentIdToBookhoardUUID in all handlers
- Add ContentType detection for Kobo EPUB/PDF sync (EPUB=6, PDF=5)
- Add "last-read-place" bookmark type support with EPUB CFI position tracking
- Restore broken mapContentIdToBookhoardUUID function with UUID parsing
- Restore mapBookhoardUUIDToKoboContentId helper function
- Restore getCollectionMetadataForBook helper function
This fixes the catastrophic file corruption from commit 2200720 which
deleted 414 lines and inserted code in the wrong location.
Phase 1-3 of KOBO_IMPLEMENTATION_PLAN.md completed:
- Step 4: Nil UUID checks in Markup, Bookmark, AnalyticsGettests, SyncFromServer
- Step 5: ContentType field added to KoboReadingSync struct
- Step 6: last-read-place case added to Markup handler switch statement
Testing: Code compiles successfully, all handlers properly structured
- Fix string(rune(remaining)) to strconv.Itoa(remaining) in device_auth.go
- Prevents garbage characters in X-RateLimit-Remaining header
- No functionality changes, only fixes broken headers
Testing: Verified with code inspection that headers return proper integers
- Fix TestUpdateDevice: Use correct JSON field name and handle float64 type
- Fix TestRejectDeviceRegistration: Expect message response instead of boolean
- Update approve device docs: Add missing response fields
- Update reject device docs: Correct message text and format
- Update Bruno API: Fix example response for reject endpoint
Both integration tests now pass while maintaining API consistency.
System Settings Bruno Tests (new):
- bruno/system/get-scan-settings.bru
- bruno/system/update-scan-settings.bru
- Test GET endpoint for retrieving system scan settings
- Test PUT endpoint for updating system scan settings
- Include admin authentication requirements
- Document response structures
List Users Bruno Test (update):
- bruno/user/admin/List Users.bru
- Add max_devices to response documentation
- Add device_count to response documentation
- Update feature descriptions
These Bruno tests provide API contract verification for the new
system settings endpoints and document the enhanced user list response.
System Settings API Documentation (new):
- docs/developer/api/system/settings.md
- Document GET /api/libraries/scan-settings endpoint
- Document PUT /api/libraries/scan-settings endpoint
- Include request/response examples
- Document validation rules and error codes
- Include migration notes from per-user to system-wide
User List API Documentation (update):
- docs/developer/api/admin/list_users.md
- Add max_devices field to response
- Add device_count field to response
- Include complete response field descriptions table
- Update example to show new fields
Documentation covers both the new system-wide scan settings feature
and the enhanced user list with device monitoring capabilities.
System Settings Tests (new file):
- Create system_settings_test.go with comprehensive test coverage
- Test admin-only access control
- Test validation (15-1440 minute range)
- Test error handling scenarios
- Test integration with scheduler
User Tests Cleanup:
- Remove old TestScanSettings from user_test.go
- Scan settings moved to system-wide (no longer per-user)
Device Cap Tests Enhancement:
- Update TestListUsersIncludesMaxDevices
- Add assertion for device_count field
- Verify both max_devices and device_count in response
All tests verify the migration from per-user to system-wide scan settings.
Add SystemSettingsHandler initialization in main.go:
- Create systemSettingsHandler instance with queries
- Add to router.Config for route registration
- Properly wired with existing dependencies
This enables the system settings endpoints to be registered and functional.
Update scheduler to use system-wide settings instead of per-user:
- Change Database interface to use GetSystemSetting
- Remove GetScanSettings (per-user method)
- Update checkAndScheduleScans to read system settings
- Apply system-wide scan frequency to all libraries
Scheduler now respects global scan settings for all library scanning,
enabling consistent system-wide scan behavior.
Clean up auth.go after migrating to system-wide settings:
- Remove UpdateScanSettings handler (moved to system_settings.go)
- Remove GetScanSettings handler (moved to system_settings.go)
- Remove UpdateScanSettingsRequest type (now in system_settings.go)
These handlers are now in SystemSettingsHandler with system-wide scope
instead of per-user functionality.
Create new SystemSettingsHandler for managing system-wide scan settings:
- GetScanSettings: retrieve scan frequency and auto-scan status
- UpdateScanSettings: update scan settings (15-1440 minutes range)
- Admin-only access (no user-specific data)
- Key-value based storage instead of per-user settings
Replaces per-user scan settings with centralized system configuration.
This handler is used by /api/libraries/scan-settings endpoints.
Auto-generated changes from running 'sqlc generate' after query updates:
- models.go: updated with SystemSettings struct, removed scan fields from Users
- querier.go: updated interface with new system settings methods
- queries.sql.go: regenerated with new query methods
Generated via: cd internal/database && sqlc generate
john-okeefe