Add comprehensive documentation for Phase 4.6 which enables the
CreateCollection endpoint to support manual book selection alongside
auto-assign rules. This is required for the Custom Section Builder.
Key additions:
- Phase 4.6: Update CreateCollection Endpoint (30-45 min)
- Add ManualBookIDs field to CreateCollectionRequest struct
- Implement graceful handling of invalid book IDs
- Add validation (max 50 book IDs) to prevent DoS
- Reuse existing AddBookToCollection service method
- Maintain backward compatibility (field is optional)
- Updated Phase 12.5: Collections Bruno tests
- create-collection-with-manual-books.bru
- create-collection-too-many-books.bru (validation test)
- create-collection-invalid-book-id.bru
- create-collection-rules-only.bru
- create-collection-unauthorized.bru
- Added section 13.3: Collections API documentation
- manual_book_ids field documentation
- Validation limits (max 50 items)
- Example combining auto-assign + manual books
- Error handling explanation
Design decisions:
- Graceful degradation: Collection created even if some books fail
- Reuse existing infrastructure: No new service methods needed
- Backward compatible: Optional field doesn't break existing clients
- UI constraint: 50 book limit prevents abuse while allowing flexibility
Updated Carousel Dashboard documentation to reflect finalized architecture decisions
and resolve discrepancies between plan and verification checklist.
## CAROUSEL_DASHBOARD_PLAN.md Changes
### Added Phase 4.5: Collections Preview Endpoint
- Documented why preview endpoint is required (web UI + mobile apps)
- Explained why client-side preview is a bad idea (download entire library,
code duplication, maintenance nightmare)
- Added full PreviewCollection handler implementation
- Added Bruno test specification
### Enhanced Phase 7: Router Registration & Config Setup
- Renamed from "Router Registration" to "Router Registration & Config Setup"
- Added Step 1: Update router.go Config struct with line numbers
- Added Step 2: Update main.go initialization with line numbers
- Added Step 3: Update test_helpers.go with line numbers
- Added explanation: Why both DashboardService AND DashboardHandler?
### Updated Phase 10.5.4: Collections Preview Endpoint
- Referenced Phase 4.5 (endpoint already implemented earlier)
- Clarified needed for web UI AND mobile apps
- Noted no additional work needed
### Added Phase 10.6: Implementation Checklist
- 30+ checklist items with file paths and verification commands
- Organized by layer (Database, Service, Handler, Router, Templates, TypeScript, Tests, Docs)
- Added Build & Verification section
- Added Timeline Estimate (20-26 hours)
- Added Post-Implementation Tasks
## CAROUSEL_DASHBOARD_VERIFICATION_CHECKLIST.md Changes
### Added Clarification Section (at top)
- Explained all discrepancies between plan and checklist
- Preview endpoint IS in plan (Phase 4.5)
- Custom Section Builder IS in plan (Phase 10.5.2 and 10.5.3)
- Service method names - Plan is correct
- Config struct - Documented with exact line numbers
- DashboardService vs DashboardHandler - Explained why both needed
### Updated Service Method Names (Section 3.2)
Changed to match plan's actual implementation:
- GetDashboardSections (not GetSectionItems)
- filterHiddenCollections (not filterHiddenSections)
- reorderCollections (not reorderSections)
- sortByPriority (new method)
- getUserCollectionItems (not getCollectionSections)
- getCollectionItemsByQueryType (renamed)
### Enhanced Config Verification (Section 6.4)
Added exact line numbers for all 3 files:
- internal/router/router.go lines 58-59
- cmd/server/main.go lines 123-124, 172-173
- cmd/server/tests/test_helpers.go lines 419-420, 458-459
### Updated Preview Endpoint Section (Section 6.3)
Added clear explanation of why endpoint is REQUIRED and why NOT client-side.
### Clarified Custom Section Builder (Sections 8.5, 9.4)
Both now explicitly state "IS in the plan (Phase 10.5)"
## docs/developer/api/dashboard.md Changes
Updated API documentation to match new unified collections architecture:
- Terminology: "smart sections" → "system collections"
- Field: `type: string` → `is_system: boolean`
- Field: `id` → `media_item_id` for books
- Request: `hidden_sections` → `hidden_collections`
- Request: `section_order` → `collection_order`
- Removed: "in-progress" and "unread" smart sections
- Added: Update Dashboard Preferences endpoint
- Added: Restore System Collection endpoint
- Updated: Example responses with new field names and types
- Updated: Error responses table
## Impact
These changes clarify:
1. Preview endpoint is required for both web UI custom section builder and mobile apps
2. Custom Section Builder IS a major feature in the plan (not missing)
3. Service method names use "collections" terminology consistently
4. Config struct updates are clearly documented with exact line numbers (3 files only)
5. Why both DashboardService AND DashboardHandler are needed in Config
All documentation now accurately reflects the finalized Carousel Dashboard architecture.
UPDATES:
- Remove smart_section_types table references
- Update for collections table with user_id, query_type, priority, is_system_collection
- Update TypeScript type examples (8 fields instead of 11)
- Update type field values ('system'/'user' instead of 'smart'/'collection')
- Update method names: getContinueReading, getNotStarted, RestoreSystemCollection
- Update field names: hidden_collections, collection_order
- Update template verification for collection terminology
- Add per-collection restore button verification
- Remove getInProgress and getUnread method references
- Update all example code to match unified architecture
VERIFICATION:
- All checklist items now verify unified collections approach
- Type examples show correct 8-field structure
- System collections properly distinguished from user collections
- Per-collection restore functionality included
BREAKING CHANGES:
- Remove smart_section_types table entirely
- Use collections table for both system defaults and user sections
- Add user_id (nullable), query_type, priority, is_system_collection to collections
- Pre-seed 4 system collections (user_id = NULL): continue-reading, recently-added, recently-read, not-started
FEATURES:
- System collections are now editable by users
- Per-collection restore functionality (restore-system-collection endpoint)
- Single query type for all dashboard items (unified approach)
UPDATES:
- Database schema changes for collections table
- Service layer methods updated (RestoreSystemCollection instead of RestoreSystemCollections)
- API handler with per-collection restore endpoint
- Templates updated with collection terminology
- TypeScript types updated (8 fields instead of 11, type values: 'system'/'user')
- Field names updated: hidden_collections, collection_order
- All tests updated for new architecture
BENEFITS:
- Simpler data model (single table, single concept)
- System defaults use same code path as user collections
- Users can customize system collections
- Easy reset with per-collection restore buttons
Major updates:
- Reduce smart sections from 5 to 4 (removed 'In Progress')
- Continue Reading: 0% < progress < 100%
- Recently Added: newest items
- Recently Read: progress >= 100%
- Not Started: progress = 0% or no record
- Update paths from web/ts/ to web/src/ structure
- Add handler types instead of duplicate template types
- Types defined in internal/handlers/dashboard.go
- Templates import handlers.SectionData, handlers.BookInfo directly
New features:
- Drag-and-drop section reordering
- Section visibility toggles
- Items per section slider
- Manual progress marking (mark as read/unread)
TypeScript updates:
- Use (window as any).api from web/src/api.ts
- Use (window as any).showToast from web/src/toast.ts
- Import types from web/src/types/dashboard.d.ts
- Event delegation via data-action attributes
Add verification checklist for comprehensive plan review:
- Type definition verification against actual API responses
- API contract and endpoint verification
- Cross-reference verification for template-handler types
- Progressive enhancement testing
- Build and deployment verification
- Compiled from web/src/search.ts
- Added proper type annotations
- Fixed null checks for DOM elements
- Added escapeHtml for library name display
- Updated onclick to use window.selectLibraryAndBook
Fixed integration tests to use existing helpers from test_helpers.go:
Changes:
- Replace getUserUUIDFromToken() with getTestUserID(t, db) helper ✅
- Replace parseUUID() with uuid.MustParse() ✅
- Add explicit comments about automatic cleanup via t.Cleanup() ✅
Test Helpers Used (all from test_helpers.go):
- setupTestServer(t) - creates test server with automatic cleanup
- loginTestUser(t, ts, db) - logs in admin user
- loginRegularUser(t, ts, db) - logs in regular user
- setupDeviceTest(t) - creates server + user + device + library
- getTestUserID(t, db) - gets/creates admin test user UUID
- uuid.MustParse() - parses UUID strings
Cleanup Pattern:
- Automatic via t.Cleanup() inside setupTestServer()
- Registered automatically when setupTestServer() is called
- No manual defer setup.Close() needed
- Runs even if test fails or panics
- Cleanup order: queue → connections → server → database
Dashboard-Specific Helper:
- updateDashboardPreferences() - only for dashboard testing
- Saves dashboard preferences for test scenarios
Benefits:
- Uses proven, existing helpers (no reinventing the wheel)
- Automatic cleanup prevents resource leaks
- Follows project testing patterns exactly
- Less custom code = fewer bugs
Added comprehensive documentation of available test helpers:
Available Helpers (from test_helpers.go):
- setupTestServer(t) - Creates test server with auto cleanup via t.Cleanup()
- loginTestUser(t, ts, db) - Logs in admin user, returns JWT token
- loginRegularUser(t, ts, db) - Logs in regular user, returns JWT token
- setupDeviceTest(t) - Creates server + user + device + library
- getTestUserID(t, db) - Gets/creates admin test user UUID
- getRegularUserID(t, db) - Gets/creates regular test user UUID
TestServerSetup Structure:
- Server *httptest.Server
- DB *database.Queries
- DBPool *pgxpool.Pool
- Config *config.Config
- ConnManager, QueueProcessor
- Auto cleanup via t.Cleanup()
Cleanup Pattern:
- Automatic cleanup registered in setupTestServer()
- Runs even if test fails or panics
- Order: queue processor → connection manager → HTTP server → database pool
- No manual defer setup.Close() needed
Updated Integration Tests:
- Added proper imports (database, uuid, pgtype)
- Documented available helpers
- Removed custom helpers that don't exist
- Uses existing project patterns
This ensures developers know what helpers are available and how to use them correctly.
Major architectural improvements:
1. Add generic /api/dashboard/sections JSON endpoint
- Created internal/handlers/dashboard.go (new file)
- Created internal/router/dashboard.go (new file)
- Single source of truth for web UI, mobile apps, plugins
- Follows existing handler/router pattern
2. Update DashboardService to apply user preferences
- GetSectionItems() now accepts sectionOrder and hiddenSections
- filterHiddenSections() removes user's hidden sections
- reorderSections() applies user's custom order
- Ensures consistent behavior across all clients
3. Separate concerns properly
- API handlers in internal/handlers/dashboard.go
- SSR routes remain in internal/router/frontend.go
- Both use same DashboardService (single source of truth)
4. Reorganize implementation phases
- Phase 1-3: Database, service, queries
- Phase 4-6: Handler, router, frontend routes
- Phase 7-9: Templates and settings
- Phase 10-11: TypeScript modules
- Phase 12-13: Documentation and testing
5. Add documentation
- docs/developer/api/dashboard.md (API reference)
- docs/user/dashboard.md (user guide)
6. Bruno tests already exist
- bruno/dashboard/ has 5 comprehensive test files
- Three-context testing (no user, user, admin)
- No additional tests needed
Benefits:
- Uniform dashboard across web, mobile, plugins
- Single source of truth (no duplicate logic)
- User preferences respected by all clients
- Follows established project patterns
- Comprehensive test coverage
Timeline: Updated to reflect 13 phases (19-25 days total with TypeScript)
- Add prerequisites section (TypeScript conversion must be completed first)
- Update execution order: TypeScript (16-21 days) then dashboard (3-4 days)
- Revise Phase 7 TypeScript implementation:
- Change file locations from web/src/ to web/ts/features/dashboard/
- Replace inline onclick with data-action attributes
- Use shared apiClient instead of raw fetch()
- Add event delegation with on() utility
- Import showToast from core instead of global
- Add type definitions matching Go handlers
- Update templates to load new TypeScript module paths
- Add summary of key changes from original plan
- Ensure consistency with TypeScript Conversion Plan patterns
- Create comprehensive plan to convert ~5,500 lines of inline JS to TypeScript
- Hybrid SSR + TypeScript CRUD approach (keeps existing JSON API)
- Event delegation pattern (no inline onclick handlers)
- Shared infrastructure: apiClient, toast, event utilities
- Procedural/imperative style (no OOP, classes, inheritance)
- 6 phases, 16-21 day timeline
- Preserves single API for all clients (web, mobile, plugins)
- No new backend routes needed
Major restructuring of the dashboard implementation plan to better match project guidelines:
- Change from handler types to template types (SectionData, BookCardData)
- Update from TypeScript to inline JavaScript matching existing pattern
- Change from separate handlers to inline routes in frontend.go
- Refactor service to return raw data (handler formats for templates)
- Add Settings template implementation
- Update TypeScript files to use IIFE pattern with window exports
- Change from .bru files to OpenCollection YAML .yml files
- Add Bruno tests note indicating tests already exist in bruno/dashboard/
This aligns the plan with actual project patterns and reduces architectural divergence.
Add after-response scripts to automatically save access and refresh tokens to Bruno environment variables after successful authentication. This eliminates manual token copying during testing.
Changes:
- Refresh Token.yml: Add script to save tokens from refresh response
- Register User.yml: Add script to save tokens from registration response
- Move all files from bruno-yaml/* to bruno/*
- Maintains existing directory structure within categories
- Updates bruno/user/auth files with OAuth2 refresh token flow
- Updates bruno/user/profile files for user profile management
- Adds bruno/dashboard/ directory with dashboard API tests
- Preserves all existing test scenarios and OpenCollection YAML format
- No functional changes - file reorganization only
- Add seven_day_session_test.go with comprehensive test coverage:
- Test login returns 7-day session (expires_in: 604800)
- Test cookie MaxAge is 7 days (604800 seconds)
- Test refresh token returns 7-day access token
- Test JWT token has 7-day expiration claim
- Test 401 error handler redirects HTML requests
- Test 401 error handler returns JSON for API requests
- Test register/login do not set document.cookie
- Tests use getTestUserID() and setupTestServer() helpers
- Update security_test.go JWT expiration comment to reflect 7 days
Tests verify all aspects of the 7-day session implementation
including constants usage, cookie values, API responses, and
smart 401 error handling.
- Update fetch interceptor to special-case 401 responses
- Clear invalid tokens from localStorage on 401 (token, refreshToken, user)
- Distinguish between page navigation and API calls:
- Page navigation: throw error to prevent further processing
- API calls: show toast error with session expired message
- Suppress network error toast for redirect errors
- Compile TypeScript to JavaScript
This ensures frontend properly handles expired sessions by clearing
stale credentials and showing appropriate error messages.
- Update Login template to accept sessionExpired boolean parameter
- Add conditional message box when session=expired query param present
- Update /login route handler to parse session query param
- Pass sessionExpired flag to Login template
- Regenerate login_templ.go with new signature
Displays friendly message: "Your session has expired. Please log in
again to continue." when users are redirected due to expired sessions.
- Add strings import for Accept header parsing
- Add wantsHTML() helper function to detect HTML vs API requests
- Checks Accept header for text/html
- Checks HX-Request header for HTMX requests
- Checks X-Requested-With for AJAX (should return JSON)
- Defaults to JSON for API routes
- Update JWT middleware ErrorHandler to:
- Redirect HTML requests to /login?session=expired
- Return JSON error for API requests with session_expired message
- Enables browser navigation to redirect gracefully while API calls
return proper error responses
This fixes the issue where protected routes returned JSON 401
for browser navigation instead of redirecting to login.
- Add SessionDuration constant (7 days) and SessionDurationSec computed value
- Update JWT token expiration to use SessionDuration instead of 1 hour
- Update register/login cookie MaxAge to use SessionDurationSec (604800)
- Update register/login API response ExpiresIn to use SessionDurationSec
- Update refresh token endpoint ExpiresIn to use SessionDurationSec
- Remove redundant client-side document.cookie lines from login/register
- Add TODO comment for HTTPS cookie Secure flag
This provides Google-like persistent sessions with a single source of truth
for session duration, eliminating hardcoded values throughout the codebase.
- Add Bearer token from localStorage to regenerate-token API request
- Update code formatting for consistency (double quotes, indentation)
This ensures the device token regeneration endpoint receives proper
authentication via the Authorization header.
- Remove 'auth: inherit' from POST request configuration
- Remove post-response script that set token environment variable
- Clean up documentation formatting
This simplifies the login request configuration as authentication
will now be handled via HTTP-only cookies instead of bearer tokens.
- Set HTTP-only cookie in login handler for SSR authentication
- Set HTTP-only cookie in registration handler
- Change default redirect from /bookshelf to /dashboard
- Cookie enables browser page navigation without JavaScript
- Add ScannerHandler field to Config struct for frontend route access
- Move scannerHandler creation before registerFrontendRoutes call
- Enables /progress page to access scanner data
- Update header navigation to link to /dashboard instead of /bookshelf
- Update index page auto-redirect to use /dashboard
- Remove duplicate route, keeping full-featured dashboard with filters
- Serve static files from web/static directory
- Add theme class safelist to Tailwind config for dynamic theming support
- Regenerate CSS with updated configuration
john-okeefe