Phase 10.5.1: Add /custom-section frontend route
- Added route handler in internal/router/frontend.go
- Fetches user libraries and renders custom section builder template
Phase 10.5.2: Create custom section builder template
- Created templates/custom_section.templ with full UI
- Includes section details form, filter rules builder, manual book selection
- Live preview functionality with preview container
- Form actions for save/cancel
Phase 10.5.3: Create custom-section-builder TypeScript
- Created web/src/custom-section-builder.ts with 13+ filter fields
- Filter fields: title, author, genre, series, progress, rating, date_added, last_read, publisher, language, format, tags, narrators
- Procedural/imperative style (no OOP) as per guidelines
- Rule builder with AND/OR logic support
- Book search and multi-select functionality
- Live preview via /api/collections/preview endpoint
- Form validation and submission to /api/collections
Phase 10.5.4: Build TypeScript modules
- Compiled custom-section-builder.ts to web/static/custom-section-builder.js
- Verified successful compilation with no errors
- All existing TypeScript modules continue to compile
Phase 10.5.5: Add Bruno tests for custom section creation
- create-custom-section-rules.bru: Test creating section with filter rules
- create-custom-section-manual.bru: Test creating section with manual book selection
- create-custom-section-missing-fields.bru: Test error handling for missing required fields
Phase 10.6: Build Verification
- ✅ TypeScript modules compile successfully
- ✅ Templates generate successfully
- ✅ Go build succeeds with no compilation errors
- ✅ All build artifacts verified (dashboard.js, custom-section-builder.js, dashboard_templ.go, custom_section_templ.go)
This completes the Custom Section Builder feature, allowing users to create
personalized dashboard sections with flexible filter rules or manual book selection.
Add preview endpoint for custom section builder and rule evaluation:
Handler Implementation (internal/handlers/collections.go):
- PreviewCollection method: Evaluates filter rules and returns matching items without saving
* Accepts library_id, rules array, manual_book_ids array, and limit
* Evaluates rules against all library items using collectionService.EvaluateRules
* Adds manually selected books to results
* Deduplicates manual books (avoids adding same book twice)
* Applies limit (default: 20, max: 100)
* Returns array of BookInfo with matching items
- Helper function: mediaItemsToListMediaItemsRow
* Converts database.MediaItems to database.ListMediaItemsRow
* Required for EvaluateRules which expects ListMediaItemsRow type
Route Registration (internal/router/collections.go):
- POST /api/collections/preview
- Protected by JWT middleware
- Part of collections API group
Why This Endpoint is Necessary:
- Allows users to see what books match their filter rules BEFORE saving
- Avoids creating incorrect collections
- Enables testing different rule combinations quickly
- Reuses existing service logic (collectionService.EvaluateRules)
- Client-side preview would require downloading entire library (10,000+ books)
- Would duplicate 500+ lines of rule evaluation logic in TypeScript
- Would create maintenance nightmare keeping Go and TypeScript in sync
Bruno Test (bruno/collections/preview-collection.bru):
- Tests POST /api/collections/preview endpoint
- Validates status 200 response
- Validates items array in response
- Example request with genre filter rule
This endpoint is required for both the web UI Custom Section Builder and future mobile apps.
Update /dashboard route in frontend.go to use unified collections architecture:
Route Changes:
- Use DashboardService to fetch user dashboard preferences
- Get all dashboard sections (system + user collections)
- Pass sections and library data to template
- Support library_id query parameter for library switching
- Default to first visible library if no library_id specified
Service Integration:
- cfg.DashboardService.GetDashboardPreferences: Fetch user preferences
* hidden_collections: Collections to hide from dashboard
* collection_order: Custom collection ordering
* items_per_section: Number of items per collection
- cfg.DashboardService.GetDashboardSections: Fetch all sections
* System collections (user_id = NULL): continue-reading, recently-added, recently-read, not-started
* User collections: User-created collections marked for dashboard
* Applies user preferences: filters hidden, reorders, sorts by priority
- handlers.BuildSections: Convert service types to handler types
Data Flow:
1. Get user template data with theme
2. Get library_id from query param or default to first library
3. Fetch user dashboard preferences
4. Fetch dashboard sections with preferences applied
5. Convert to handler types for template rendering
6. Render template with sections and library data
Template Signature Change:
- OLD: templates.Dashboard(user)
- NEW: templates.Dashboard(user, sections, libData, currentLibraryID)
This implements Phase 8: SSR Template Routes with unified collections architecture.
Add DashboardService and DashboardHandler to application configuration:
Router Config Updates (internal/router/router.go):
- Add services import for DashboardService type
- Add DashboardService field to Config struct
- DashboardService: Used by SSR routes in frontend.go for data fetching
- DashboardHandler: Used by API routes in dashboard.go for JSON endpoints
Server Initialization (cmd/server/main.go):
- Create dashboardService instance using services.NewDashboardService(queries)
- Keep dashboardHandler creation (already exists from Phase 4)
- Add DashboardService to routerConfig
- Both services now available for dependency injection
Test Helpers (cmd/server/tests/test_helpers.go):
- Create dashboardService instance for testing
- Create dashboardHandler instance for testing
- Add both DashboardService and DashboardHandler to routerConfig
- Ensures test environment matches production setup
Architecture Rationale:
- DashboardService: Service layer with business logic (reusable by SSR, mobile)
- DashboardHandler: HTTP handler layer (JSON API endpoints)
- Separation allows SSR templates to call service directly
- API routes use handler for proper HTTP response handling
- Mobile apps can use API endpoints via DashboardHandler
All three files updated consistently for complete integration.
Add dashboard route registration and wire up handler:
Router Changes:
- Add DashboardHandler to router.Config struct
- Create internal/router/dashboard.go with dashboard route registration
- Register dashboard routes in main RegisterRoutes function
Dashboard Routes (all protected by JWT):
- GET /api/dashboard/sections: Get dashboard sections for user
* Query params: library_id (required), limit (optional, default 20, max 100)
* Returns: JSON with sections array
- PUT /api/dashboard/preferences: Update dashboard preferences
* Body: library_id, hidden_collections, collection_order, items_per_section
* Returns: Updated preferences
- POST /api/dashboard/restore-system-collection: Restore system collection to defaults
* Body: collection_name (must be valid system collection)
* Returns: Success message
Server Integration:
- Create dashboardHandler in cmd/server/main.go
- Add dashboardHandler to routerConfig
- Routes are automatically registered on server startup
Add dashboard API endpoints with handler layer:
Step 1: Add SectionData to collections.go
- SectionData struct represents dashboard section (carousel of books)
- Used by: Dashboard handler, Templates (SSR), API JSON responses
- Shared type from collections.go (no duplicate definitions)
- Fields: ID, IsSystem, Title, Description, Icon, Items, ViewAllURL, Priority
Step 2: Create dashboard.go handler
- DashboardHandler struct with injected database and dashboard service
- GetSections: Returns dashboard sections as JSON (mobile apps, web UI TypeScript, plugins)
* Validates library_id parameter
* Fetches user dashboard preferences
* Configurable limit (default 20, max 100)
* Calls service layer for business logic
* Converts service types to handler types for JSON serialization
- UpdatePreferences: Saves dashboard preferences
* Validates library_id
* Upserts user dashboard preferences
- RestoreSystemCollection: Resets system collection to defaults
* Validates collection_name against allowed system collections
* Deletes user's copy (system collection reappears automatically)
- BuildSections: Converts service DashboardSection to handler SectionData
* Converts database.MediaItems to handlers.BookInfo
* Uses shared types from collections.go
- getViewAllURL: Maps system collections to their view-all URLs
- Reuses existing textToString helper from collections.go
Architecture Compliance:
- Generic API handler for reuse by SSR, mobile, plugins
- Uses shared types from collections.go (SectionData, BookInfo)
- IsSystem bool matches database field (no string conversion)
- Single service method returns structured data (simpler, less bugs)
- Handler just converts types (no matching logic needed)
- Reusable by mobile apps, web UI, plugins
Create DashboardService with business logic for Carousel-style dashboard:
Service Methods:
- NewDashboardService: Create service instance with injected dependencies
- GetDashboardSections: Fetch all collections (system + user) with their items
* Gets system collections (user_id = NULL) by query type
* Gets user collections with manual + auto-assigned items
* Filters hidden collections based on user preferences
* Reorders collections based on user custom order
* Sorts by priority if no custom order exists
- GetDashboardPreferences: Fetch user dashboard preferences for library
- UpsertDashboardPreferences: Save or update user dashboard preferences
- RestoreSystemCollection: Reset user's copy of system collection to defaults
Helper Methods:
- filterHiddenCollections: Remove hidden collections from results
- reorderCollections: Reorder sections based on user preference
- sortByPriority: Sort sections by priority (lower numbers first)
- getCollectionItemsByQueryType: Return items for system collections by query type
- getUserCollectionItems: Return items for user collections (manual + auto-assign)
Type Conversion Helpers:
- mediaItemsToListMediaItemsRow: Convert MediaItems to ListMediaItemsRow for rule evaluation
- getCollectionItemsRowToMediaItems: Convert GetCollectionItemsForDashboardRow to MediaItems
Architecture Compliance:
- Service layer holds all business logic (reusable by SSR, API, mobile)
- Returns database types (type safety at DB layer)
- Handler converts to API types (clean JSON contracts)
- Uses existing database queries and collection service
- Procedural/imperative style (no OOP)
- Follows existing pattern from collections.go
Add SQL queries for dashboard functionality and system collections:
Dashboard Preferences Queries:
- GetDashboardPreferences: Fetch user preferences for a library
- UpsertDashboardPreferences: Create or update user dashboard preferences
- UpdateDashboardPreferences: Update existing preferences
Dashboard Collections Queries:
- GetSystemCollectionsForDashboard: Fetch system collections (user_id IS NULL)
- GetUserCollectionsForDashboard: Fetch user collections marked for dashboard
- DeleteUserSystemCollection: Delete user's copy of a system collection
System Collection Smart Queries:
- GetContinueReadingItems: Books with 0 < progress < 1
- GetRecentlyAddedItems: Newly added items to library
- GetRecentlyReadItems: Books with progress >= 1
- GetNotStartedItems: Books with progress = 0 or no record
Collection Management Queries:
- GetCollectionItemsForDashboard: Fetch collection items with excluded flag
- GetLibraryItems: Fetch all items in a library
These queries support the unified collections architecture where system
defaults and user-created sections are both collections with user_id
NULL for system-owned and NOT NULL for user-created.
Add support for Carousel-style dashboard with unified collections architecture:
Database Schema Changes:
- Add user_dashboard_preferences table:
- hidden_collections: TEXT[] for managing section visibility
- collection_order: TEXT[] for custom ordering
- items_per_section: INT for limiting items per section
- Update collections table:
- user_id: Make nullable to support system-owned collections (NULL = system)
- show_on_dashboard: BOOLEAN for controlling visibility
- query_type: TEXT for different query types (continue-reading, recently-added, etc.)
- priority: INT for display order (lower = higher priority)
- is_system_collection: BOOLEAN for flagging system defaults
- Update collection_items table:
- Add excluded BOOLEAN for user overrides of auto-assigned items
Indexes:
- idx_collections_dashboard: (user_id, show_on_dashboard, priority) WHERE show_on_dashboard = true
- idx_dashboard_prefs_user_library: (user_id, library_id)
- idx_collection_items_excluded: (collection_id, excluded) WHERE excluded = true
System Collections (pre-seeded defaults):
- continue-reading: Books with 0 < progress < 1
- recently-added: Newly added items to library
- recently-read: Books with progress >= 1
- not-started: Books with progress = 0 or no record
This implements Phase 1 of the Carousel-style dashboard redesign plan.
- Update Login template to accept sessionExpired boolean parameter
- Add conditional message box when session=expired query param present
- Update /login route handler to parse session query param
- Pass sessionExpired flag to Login template
- Regenerate login_templ.go with new signature
Displays friendly message: "Your session has expired. Please log in
again to continue." when users are redirected due to expired sessions.
- Add strings import for Accept header parsing
- Add wantsHTML() helper function to detect HTML vs API requests
- Checks Accept header for text/html
- Checks HX-Request header for HTMX requests
- Checks X-Requested-With for AJAX (should return JSON)
- Defaults to JSON for API routes
- Update JWT middleware ErrorHandler to:
- Redirect HTML requests to /login?session=expired
- Return JSON error for API requests with session_expired message
- Enables browser navigation to redirect gracefully while API calls
return proper error responses
This fixes the issue where protected routes returned JSON 401
for browser navigation instead of redirecting to login.
- Add SessionDuration constant (7 days) and SessionDurationSec computed value
- Update JWT token expiration to use SessionDuration instead of 1 hour
- Update register/login cookie MaxAge to use SessionDurationSec (604800)
- Update register/login API response ExpiresIn to use SessionDurationSec
- Update refresh token endpoint ExpiresIn to use SessionDurationSec
- Remove redundant client-side document.cookie lines from login/register
- Add TODO comment for HTTPS cookie Secure flag
This provides Google-like persistent sessions with a single source of truth
for session duration, eliminating hardcoded values throughout the codebase.
- Set HTTP-only cookie in login handler for SSR authentication
- Set HTTP-only cookie in registration handler
- Change default redirect from /bookshelf to /dashboard
- Cookie enables browser page navigation without JavaScript
- Add ScannerHandler field to Config struct for frontend route access
- Move scannerHandler creation before registerFrontendRoutes call
- Enables /progress page to access scanner data
- Serve static files from web/static directory
- Add theme class safelist to Tailwind config for dynamic theming support
- Regenerate CSS with updated configuration
- Add LibraryTestData struct to TestDeviceSetup
- Implement CreateLibrary() for proper library creation in tests
- Implement CreateCollection() for test collection support
- Improve test isolation with dedicated library creation
This provides a more robust foundation for integration tests that need
proper library management support.
- Display sync URLs for Kobo devices with copy button
- Display auth tokens for KOReader devices with copy button
- Add regenerate token button with confirmation
- Show warning about token invalidation
- Add handler to regenerate device auth tokens
- Add PUT /api/devices/:id/regenerate-token route
- Returns new token and sync URLs for device configuration
- Apply DeviceAuthMiddleware.Authenticate to /opds/devices/* routes
- OPDS now uses same authentication model as sync API (devices.auth_token)
- Removes security vulnerability allowing unauthorized device enumeration
- Update test expectations to require 401 for unauthenticated requests
- Fix query parameter name from 'query' to 'q' in search endpoints
- Update router comments to clarify authentication requirements
- Allow media items to be created/updated with invalid ISBN by storing empty string
- Fix test to use valid ISBN-13 format (9780306406157)
- Add small delay to prevent race condition in pagination test
Enhance NormalizeISBN to validate and convert ISBNs:
- Validate length (10 or 13 digits), return error if invalid
- Convert ISBN-10 to ISBN-13 by prefixing '978' and recalculating checksum
- Add NormalizeISBNSafe for backward compatibility in scanners
This ensures all ISBNs stored in database are valid ISBN-13 format.
Add strict validation to return 400 Bad Request when any media_item_id
is empty in the bulk-update request, rather than treating it as a
partial failure with 200 OK.
This aligns the handler behavior with test expectations for the
BulkUpdateBooks_EmptyBookIDs test case.
- Try multiple locations for schema.sql file
- Support both local dev and containerized deployment paths
- Add informative logging when schema is loaded
- Prevent runtime.Caller issues in containers
Locations checked:
- database/schema/schema.sql (working directory)
- /app/database/schema/schema.sql (container)
- ../database/schema/schema.sql (relative)
- ../../database/schema/schema.sql (relative)
This fixes the 'no such file or directory' error in production containers.
- Create internal/database/schema.go with full initialization logic
- Parse table names from schema.sql using regex (handles both formats)
- Execute schema in atomic transaction
- Verify all expected tables exist
- Verify all critical functions exist (6 functions)
- PostgreSQL advisory locking with 30-second timeout
- Self-healing from partial/corrupted state
- Load schema.sql from filesystem at runtime
Features:
- Defensive regex handles IF NOT EXISTS and legacy CREATE TABLE
- Lock timeout prevents indefinite hangs
- Function verification ensures sync operations work
- Clear error messages with debug hints
- Add nil UUID checks after mapContentIdToBookhoardUUID in all handlers
- Add ContentType detection for Kobo EPUB/PDF sync (EPUB=6, PDF=5)
- Add "last-read-place" bookmark type support with EPUB CFI position tracking
- Restore broken mapContentIdToBookhoardUUID function with UUID parsing
- Restore mapBookhoardUUIDToKoboContentId helper function
- Restore getCollectionMetadataForBook helper function
This fixes the catastrophic file corruption from commit 2200720 which
deleted 414 lines and inserted code in the wrong location.
Phase 1-3 of KOBO_IMPLEMENTATION_PLAN.md completed:
- Step 4: Nil UUID checks in Markup, Bookmark, AnalyticsGettests, SyncFromServer
- Step 5: ContentType field added to KoboReadingSync struct
- Step 6: last-read-place case added to Markup handler switch statement
Testing: Code compiles successfully, all handlers properly structured
- Fix string(rune(remaining)) to strconv.Itoa(remaining) in device_auth.go
- Prevents garbage characters in X-RateLimit-Remaining header
- No functionality changes, only fixes broken headers
Testing: Verified with code inspection that headers return proper integers
Update scheduler to use system-wide settings instead of per-user:
- Change Database interface to use GetSystemSetting
- Remove GetScanSettings (per-user method)
- Update checkAndScheduleScans to read system settings
- Apply system-wide scan frequency to all libraries
Scheduler now respects global scan settings for all library scanning,
enabling consistent system-wide scan behavior.
Clean up auth.go after migrating to system-wide settings:
- Remove UpdateScanSettings handler (moved to system_settings.go)
- Remove GetScanSettings handler (moved to system_settings.go)
- Remove UpdateScanSettingsRequest type (now in system_settings.go)
These handlers are now in SystemSettingsHandler with system-wide scope
instead of per-user functionality.
Create new SystemSettingsHandler for managing system-wide scan settings:
- GetScanSettings: retrieve scan frequency and auto-scan status
- UpdateScanSettings: update scan settings (15-1440 minutes range)
- Admin-only access (no user-specific data)
- Key-value based storage instead of per-user settings
Replaces per-user scan settings with centralized system configuration.
This handler is used by /api/libraries/scan-settings endpoints.
Auto-generated changes from running 'sqlc generate' after query updates:
- models.go: updated with SystemSettings struct, removed scan fields from Users
- querier.go: updated interface with new system settings methods
- queries.sql.go: regenerated with new query methods
Generated via: cd internal/database && sqlc generate
System Settings Migration:
- Add GetSystemSetting query for single setting retrieval
- Add UpdateSystemSetting query for updating settings
- Add GetAllSystemSettings query for all settings
- Remove UpdateScanSettings and GetScanSettings (per-user queries)
User Query Enhancement:
- Add max_devices field to GetUser query
- Add device_count computed field to GetUser query
- Add max_devices field to ListUsers query
- Add device_count computed field to ListUsers query
These changes support:
1. System-wide scan settings instead of per-user settings
2. Users can now see their device count and limits
3. Admins can monitor device usage across all users
john-okeefe