Commit Graph
423 Commits
Author SHA1 Message Date
john-okeefe 4a19699e8f feat(templates): add types for progress, unlinked books, and shelf mappings
Add template data types for Phase 9 frontend features:
- ProgressItemData: For reading progress visualization with sync source
- UnlinkedBookData: For unmatched books requiring manual linking
- PotentialMatchData: For book matching suggestions
- DeviceShelfMappingData: For collection-to-shelf mappings

Also adds Theme field to User type for theme support.
2026-02-01 00:25:09 -05:00
john-okeefe 6266a06abb feat: Add SSR for conflicts and queue pages
- Update conflicts.templ to accept pre-rendered data
- Update queue.templ to accept pre-rendered data
- Update /conflicts and /queue routes in main.go for SSR
- Update stats rendering to use server-side values
- Add server-side conflict list rendering
- Add server-side queue list rendering
- Update conflicts.js to use location.reload() after operations
- Update queue.js to use location.reload() after operations
- Remove initial load calls from JavaScript files

Preserves all API endpoints and backward compatibility
2026-01-31 23:14:08 -05:00
john-okeefe b3c0c0c225 feat(handlers): Add GetDevicesData helper and update devices template for SSR
- Add GetDevicesData() to DeviceHandler (returns raw data, not JSON)
- Update devices template signature to accept pre-rendered data
- Add server-side rendering of devices and pending registrations
- Update JavaScript to use location.reload() after CRUD operations
- Remove getDeviceIcon dependency on JavaScript function
- Use templ if/else instead of ternary operators for device status

Preserves all API endpoints and backward compatibility
2026-01-31 22:42:21 -05:00
john-okeefe 86eaee5a25 lint(handlers): Use strings.EqualFold for case-insensitive comparison
Fix SA6005 staticcheck warning in OPDS handler
2026-01-31 22:34:48 -05:00
john-okeefe d6d208c3d1 test(bruno): Add API tests for Kobo, Koreader, OPDS, collections
- Add Kobo bookmark/markup sync tests
- Add Koreader progress/bookmark sync tests
- Add OPDS feed tests
- Add collection tests
- Add sync tests for book linking
2026-01-31 22:32:56 -05:00
john-okeefe 0a97b679a6 docs: Add hybrid SSR and project guidelines documentation
- Document hybrid SSR architecture for frontend implementation
- Add project guidelines for development workflow
- Explain API preservation and SSR approach
2026-01-31 22:32:52 -05:00
john-okeefe 033d66f503 feat(templates): Add hybrid SSR collection templates
- Add Collection template with SSR support
- Add CollectionDetail template for book display
- Templates accept pre-rendered data from server
2026-01-31 22:32:47 -05:00
john-okeefe 3d20e802e6 feat(opds): Add OPDS feed generation library
- Add OPDS feed generation for Kobo compatibility
- Support device catalog, search, download, navigation
- Add format conversion support
2026-01-31 22:32:37 -05:00
john-okeefe 964a4583ab feat(services): Add book matching and collection services
- Add book matching service for intelligent book deduplication
- Add collection service for collection management
- Add test files for book matching and collections
2026-01-31 22:32:28 -05:00
john-okeefe 6495cc2c7c feat(handlers): Add OPDS, collections, book matching, and sync handlers
- Add OPDS handler for device catalog and book downloads
- Add collections handler for collection CRUD
- Add book matching service for cross-device book linking
- Add sidecar handler for Kobo metadata sync
- Add sync handler for device synchronization
2026-01-31 22:32:23 -05:00
john-okeefe d28002e4cb feat(sync): Add Kobo/Koreader sync and conflict handling
- Add Kobo markup/sync endpoints for bookshelves
- Add Koreader progress sync with SHA-256 support
- Add sync conflict detection and resolution
- Update ebook scanner for better file matching
2026-01-31 22:32:09 -05:00
john-okeefe 63008001c6 feat(db): Add kobo_shelves and device catalog tables
- Add kobo_shelves table for Kobo device shelves management
- Add device_shelf_mappings for collection<->device shelf mappings
- Update device_catalogs with better ContentId tracking
- Add indexes for performance
2026-01-31 22:32:04 -05:00
john-okeefe 60380a043d docs: Update bruno test files 2026-01-31 22:31:54 -05:00
john-okeefe cd8f313b16 feat(handlers): Add helper methods for hybrid SSR
- Add GetPendingRegistrationsData() to DeviceHandler
- Add GetQueueData() to QueueHandler
- Add template types: DeviceData, PendingRegistrationData, ConflictData, QueueItemData
- Add collection types: CollectionData, CollectionDetailData, BookData

All changes are non-breaking and preserve API compatibility
2026-01-31 22:31:33 -05:00
john-okeefe 56a9159631 refactor: remove outdated sync implementation guide
- Remove large UNIVERSAL_SYNC_IMPLEMENTATION_GUIDE.md
- Replace with more focused implementation plan
- Update documentation structure for better maintainability
2026-01-31 18:25:41 -05:00
john-okeefe 74bf439e5e docs: refactor KOReader setup guide
- Simplify and condense KOReader configuration documentation
- Remove redundant sections and improve clarity
2026-01-31 18:25:32 -05:00
john-okeefe 4d87beb540 test: add Bruno API collections for sync features
- Add device registration and management API tests
- Add conflicts resolution API tests
- Add queue management API tests
- Add KOReader sync protocol tests
- Add Kobo sync protocol tests
2026-01-31 18:25:32 -05:00
john-okeefe 707df26453 feat: add UI templates for device and queue management
- Add devices management template
- Add conflicts resolution template
- Add queue management template
- Update admin templates with navigation links
2026-01-31 18:25:29 -05:00
john-okeefe 5a7144b98e feat: add device management and queue management routes
- Add /devices route for device management interface
- Add /conflicts route for sync conflict resolution
- Add /queue route for sync queue management
- Add comprehensive tests for device cap management
- Add test suite for queue management
2026-01-31 18:25:28 -05:00
john-okeefe 33f26f3d7d docs: add implementation plan and update documentation
- Add comprehensive implementation plan for universal sync system
- Update README with API reference and device setup guides
- Add KOBO_SETUP.md device configuration guide
2026-01-31 18:25:27 -05:00
john-okeefe ab8f4f5351 docs/Code/bookmann/IMPLEMENTATION_PLAN.md: Add comprehensive implementation plan
This plan implements:
- Universal book identification (SHA-256, UUID, ISBN, ASIN, OPF identifiers)
- Enhanced collection management with auto-assign rules
- OPDS-based wireless book delivery for Kobo, KOReader, Web, and Mobile
- Bidirectional progress sync with ContentId mapping
- Device-specific shelf mappings and configuration
- Complete database schema with 7 new tables
- 50+ documented API endpoints
- 10-week phased implementation plan

Features include:
- Cross-device book matching regardless of file paths
- Collections as device-neutral metadata with per-device shelf mapping
- Format conversion (EPUB → KEPUB) with hash integrity preservation
- Three-tier authentication (JWT, device tokens, OPDS)
- Two-layer architecture: OPDS for acquisition + internal APIs for state management

Key design principles:
1. Canonical UUID (Bookmann UUID) always wins for progress tracking
2. Collections ≠ device inventory - organizational metadata only
3. OPDS primary for all devices, internal APIs for web/mobile
4. Dual hash storage prevents format conversion issues

See IMPLEMENTATION_PLAN.md for complete technical details.
2026-01-31 18:00:28 -05:00
john-okeefe d12c5b718a Update README with sync features and device cap
- Document universal sync system
- Add device cap management feature
- Note SSL/TLS proxy configuration
- Include API endpoints and features overview
2026-01-31 13:07:09 -05:00
john-okeefe 2801cf7432 Add comprehensive documentation for sync features
- Add SECURITY_AUDIT.md with A- security rating
- Add SECURITY_ENHANCEMENTS.md for improvements
- Add DEVICE_CAP_IMPLEMENTATION.md complete guide
- Add KOREADER_SETUP.md device setup guide
- Add SYNC_USER_GUIDE.md user documentation
- Document all API endpoints and features
- Include security considerations and best practices
2026-01-31 13:07:01 -05:00
john-okeefe a7f2b83bdd Integrate sync queue system and device cap API
- Start queue processor as background goroutine
- Initialize and register queue handler
- Add queue management routes (7 endpoints)
- Update KOReader handler to use checkpoint sync mode
- Add device cap management route (PUT /api/auth/users/:id/max-devices)
- Register all new endpoints with proper middleware
2026-01-31 13:06:56 -05:00
john-okeefe 50c632babf Update SSL/TLS handling for Docker reverse proxy deployments
- Disable HTTPSRedirectMiddleware (SSL handled by proxy)
- Keep SSLProxyMiddleware for X-Forwarded-* headers
- Add note about Docker deployment architecture
- Database connections use sslmode=disable
- No redirect needed for reverse proxy setup
2026-01-31 13:06:50 -05:00
john-okeefe 31fae3cea8 Add Bruno API collection for device cap management
- Update User Max Devices - Complete API documentation
- Update User Max Devices - Success test case
- Update User Max Devices - Invalid Zero test case
- Update User Max Devices - Exceeds Maximum test case
- Update User Max Devices - Missing ID test case
- Include validation rules and example payloads
- Document all status codes and error responses
2026-01-31 13:06:46 -05:00
john-okeefe 8d55b7c5b5 Add comprehensive test suite for device cap management
- Test successful updates (5, 10, 50, 100 devices)
- Test validation failures (0, -1, 101, 1000 devices)
- Test authentication requirements (no token, non-admin)
- Test non-existent user ID
- Test missing user ID in URL
- Test max_devices field in user list response
- Add 20+ test cases across 7 test functions
- Helper functions for admin user creation and login
2026-01-31 13:06:42 -05:00
john-okeefe 5039071ea3 Add admin-configurable device cap per user
- Add max_devices column to users table (default: 10)
- Add UpdateUserMaxDevices database query
- Add CountUserDevices database query
- Add UpdateUserMaxDevices handler with validation (1-100 devices)
- Add PUT /api/auth/users/:id/max-devices endpoint (admin only)
- Update UserList struct to include max_devices field
- Validate user ID format and max_devices range
- Return appropriate errors for invalid requests
2026-01-31 13:06:21 -05:00
john-okeefe 7ead4109ef Add performance optimizations (Phase 7)
- Add 12 composite database indexes for sync operations
- Composite indexes for sync_queue (device/status/priority)
- Composite indexes for reading_progress (user/media timestamps)
- Composite indexes for devices (user/sync_enabled)
- Composite indexes for annotations (user/media)
- Comment out ALTER SYSTEM commands for sqlc compatibility
- PostgreSQL tuning recommendations included for manual application
2026-01-31 13:06:17 -05:00
john-okeefe 030d30a225 Add queue management API and database queries
- Add 7 new database queries for queue management
- GetStuckSyncQueueItems - Detect stuck items
- GetSyncQueueStats - Queue statistics
- GetNextRetryTime - Exponential backoff calc
- ListAllSyncQueueItems - Admin view
- IncrementSyncQueueAttempts - Retry counter
- Add queue handler with 7 REST endpoints
- GET /api/queue/devices/:id/stats - Queue statistics
- GET /api/queue/devices/:id/items - List device queue
- POST /api/queue/items/:id/retry - Retry failed item
- DELETE /api/queue/items/:id - Delete queue item
- DELETE /api/queue/devices/:id/clear - Clear device queue
- GET /api/queue/items - List all items (admin)
- Add full user/admin access control
2026-01-31 13:06:12 -05:00
john-okeefe 11ee4901a5 Add offline detection and recovery system (Phase 6)
- Implement OfflineDetector with 5-minute online threshold
- Add automatic device scanning (2-minute intervals)
- Add offline mode enforcement (disable sync)
- Add reconnection handling with priority item processing
- Add force reconnect API endpoint
- Add comprehensive offline detection tests
- Handle device offline/reconnected events
2026-01-31 13:06:05 -05:00
john-okeefe b9de7c48d5 Add sync queue system with exponential backoff retry logic (Phase 6)
- Implement SyncQueueProcessor with 5-second polling interval
- Add priority-based queuing (1-10 scale)
- Add exponential backoff retry logic (1m, 5m, 15m, 1h, 24h)
- Add stuck item detection (> 1 hour in processing state)
- Add batch processing (50 items per cycle)
- Add comprehensive test suite (15+ test cases)
- Test enqueue/dequeue, priority ordering, retry logic, concurrent operations
2026-01-31 13:06:00 -05:00
john-okeefe 36b54907d1 Remove test environment file with credentials
Delete .env.test which contained hardcoded credentials. Environment should be configured via .env or environment variables.
2026-01-31 11:46:07 -05:00
john-okeefe aa362a8c7c Mark gorilla/websocket as direct dependency
Promote gorilla/websocket from indirect to direct dependency as it's used for WebSocket sync functionality
2026-01-31 11:46:00 -05:00
john-okeefe 6ebe974927 Simplify docker-compose healthcheck configuration
Remove redundant volumes and env_file, simplify healthcheck to use pg_isready for both services
2026-01-31 11:45:58 -05:00
john-okeefe 5bb273f6aa Improve Phase 1 integration test robustness
Update test credentials and add better error handling for user registration and login scenarios to prevent test failures from incomplete test runs.
2026-01-31 11:45:56 -05:00
john-okeefe 2d2d643873 Add sync conflict detection and resolution system
Implement conflict detection for concurrent reading progress updates from different devices. Adds conflict management endpoints for listing, viewing, and resolving conflicts.

- Add ConflictHandler with CRUD endpoints for conflict management
- Implement automatic conflict detection in KOReader progress updates
- Add WebSocket broadcast for real-time conflict notifications
- Add database query for listing user conflicts by status
- Add integration tests and Bruno API test collection
2026-01-31 11:45:52 -05:00
john-okeefe 9b41b3ecb0 Add refresh token flow integration test
- Test login returns both access_token and refresh_token
- Test refresh endpoint accepts UUID token and returns new access_token
- Verifies end-to-end refresh token flow works correctly
2026-01-31 11:44:47 -05:00
john-okeefe d440e55f05 Update handlers to work with UUID refresh tokens
- Add parseTokenUUID() helper to convert string to pgtype.UUID
- Update RefreshAccessToken to parse token string to UUID before validation
- Update Logout to parse token string to UUID before revoking
- Update CreateRefreshToken to pass UUID directly to database
- Update auth.go: fix return value order from CreateRefreshToken
- Remove unnecessary comments for cleaner code
2026-01-31 11:44:35 -05:00
john-okeefe db5d51e77e Regenerate database code for UUID token support
- Update models: RefreshTokens.Token now pgtype.UUID
- Update querier: GetRefreshToken/RevokeRefreshToken accept pgtype.UUID
- Regenerate queries.sql.go via sqlc after schema change
2026-01-31 11:44:26 -05:00
john-okeefe 835e78898a Convert refresh_tokens.token from VARCHAR to UUID
- Change token column type from VARCHAR(255) to UUID
- Add gen_random_uuid() as default value for token
- Improves type safety and performance for token storage
2026-01-31 11:44:23 -05:00
john-okeefe 45ba922c68 test: add Bruno API test collections for Kobo and media endpoints 2026-01-31 00:29:40 -05:00
john-okeefe 42c3168fcf feat: add server sync to Kobo endpoint and media routes 2026-01-31 00:29:39 -05:00
john-okeefe fef41dc168 feat: add media handler for book download and shelf management 2026-01-31 00:29:38 -05:00
john-okeefe cd240d3054 feat: add Kobo shelf and entitlement SQL queries 2026-01-31 00:29:38 -05:00
john-okeefe ed336c6c41 feat: add Kobo entitlements and shelves models 2026-01-31 00:29:37 -05:00
john-okeefe c17b42bf83 feat: add Kobo shelf and entitlements tables to schema 2026-01-31 00:29:36 -05:00
john-okeefe 0a44bc130e chore: remove redundant test script and report files
- Remove integration_test.sh (redundant with Go integration tests)
- Remove BRUNO_PHASE1_TEST_REPORT.md (temporary test report)
- Remove integration_test_results.txt (temporary test output)

Go-based tests in cmd/server/tests/ provide comprehensive coverage and
are better integrated with the project testing infrastructure.
2026-01-31 00:08:29 -05:00
john-okeefe 3943355180 chore: add token and session file patterns to gitignore 2026-01-31 00:05:23 -05:00
john-okeefe 3c6d08a550 security: remove fresh_token.txt file from repository 2026-01-31 00:05:14 -05:00