diff --git a/EBOOK_REFACTOR_PLAN.md b/EBOOK_REFACTOR_PLAN.md deleted file mode 100644 index 7517f4e..0000000 --- a/EBOOK_REFACTOR_PLAN.md +++ /dev/null @@ -1,857 +0,0 @@ -# Comprehensive ebook.go Refactor Plan - -**Created:** February 6, 2026 -**Purpose:** Refactor 1,350-line ebook.go into focused, single-responsibility files -**Assumption:** Scanner routes already moved to router/scanner.go -**Goal:** Zero API changes, 100% functional compatibility - ---- - -## Executive Summary - -`internal/handlers/ebook.go` violates Go single responsibility principle at 1,350 lines: -- **Scanner operations**: 296 lines (12 methods) -- **Media CRUD & metadata**: 822 lines (24 methods) -- **Book matching/sync**: ~150 lines (8 methods) -- **Collection device mappings**: 60+ lines (6 methods) - -**Proposed split:** -1. `media.go` - Media items, ratings, progress, notes, highlights (CRUD + metadata) -2. `search.go` - Query, search, filtered operations -3. `matching.go` - Book linking, file aliases, matching algorithms -4. `ebook.go` - Slimmed to only core Handler struct and SetupRoutes - -**Risk Level:** VERY LOW -- No API endpoint changes -- No database schema changes -- Only code organization -- All existing tests continue to work - ---- - -## File Analysis Before Refactor - -### Current ebook.go Structure (1,350 lines) - -```go -// Lines 38-164: Handler setup and SetupRoutes() -func NewHandler(...) *Handler { } -func SetupRoutes(...) *Handler { } - -// Lines 166-461: Scanner operations (12 methods) -func (h *Handler) ScanEbooks() { } -func (h *Handler) StartScanner() { } -// ... 10 more scanner methods - -// Lines 474-1296: Media operations (24 methods) -func (h *Handler) ListMediaItems() { } -func (h *Handler) GetMediaItem() { } -func (h *Handler) CreateMediaRating() { } -// ... 21 more media methods - -// Lines 1297-end: Search & matching (8 methods) -func (h *Handler) SearchMediaItems() { } -func (h *Handler) QueryBooks() { } -// ... 6 more methods -``` - -### Handler Dependencies - -All methods share this Handler struct: -```go -type Handler struct { - db *database.Queries - worker *worker.Worker - scanner *services.EbookScanner - scheduler *cron.Cron - ctx context.Context - cancel context.CancelFunc - mu sync.RWMutex - watchingLibraries map[string]bool -} -``` - ---- - -## Phase 1: SetupRoutes Restructure (FOUNDATION) - -### Purpose -Remove all route registration from SetupRoutes, make it a pure factory function - -### File: `internal/handlers/ebook.go` - -#### Step 1.1: Simplify SetupRoutes (Lines 70-157) - -**Current Code Problem:** -```go -func SetupRoutes(g *echo.Group, db *database.Queries, connManager *wsync.ConnectionManager) *Handler { - h := NewHandler(db, connManager) - - // Collections API routes (21 lines) - collectionHandler := NewCollectionHandler(db, connManager) - collections := g.Group("/collections") - collections.GET("", collectionHandler.GetCollections) - // ... 7 more collection routes - - // Device shelf mapping routes (6 lines) - deviceCollections := g.Group("/devices/:id/collections") - // ... 6 device routes - - // Book matching routes (4 lines) - g.POST("/sync/books/query", h.QueryBooks) - // ... 3 more matching routes - - // Media item routes (4 lines) - g.GET("/media-items", h.ListMediaItems) - // ... 3 more media routes - - // Admin-only routes (11 lines) - admin := g.Group("", AdminMiddleware) - admin.POST("/media-items", h.CreateMediaItem) - // ... 10 more admin routes (including scanner) - - return h -} -``` - -**New Code:** -```go -func SetupRoutes(g *echo.Group, db *database.Queries, connManager *wsync.ConnectionManager) *Handler { - return NewHandler(db, connManager) -} -``` - -**Verification:** -- ✅ SetupRoutes now only creates handler instance -- ✅ All routes will be registered via router package -- ✅ Handler struct and dependencies unchanged - ---- - -## Phase 2: Create Media Handler (CORE CRUD) - -### File: `internal/handlers/media.go` (NEW) - -#### Step 2.1: Create MediaHandler struct - -```go -package handlers - -import ( - "bookhoard/internal/database" - "bookhoard/internal/sync" - "context" - "fmt" - "net/http" - "strconv" - "time" - - "github.com/google/uuid" - "github.com/jackc/pgx/v5/pgtype" - "github.com/labstack/echo/v4" -) - -// MediaHandler handles media item CRUD operations and metadata -type MediaHandler struct { - db *database.Queries - worker *worker.Worker -} - -// NewMediaHandler creates a new media handler -func NewMediaHandler(db *database.Queries, worker *worker.Worker) *MediaHandler { - return &MediaHandler{ - db: db, - worker: worker, - } -} -``` - -#### Step 2.2: Move media CRUD methods (From ebook.go lines 474-959) - -**Methods to move:** -- `ListMediaItems()` → `media.go:ListMediaItems()` -- `GetMediaItem()` → `media.go:GetMediaItem()` -- `ListMediaItemsFiltered()` → `media.go:ListMediaItemsFiltered()` -- `CreateMediaItem()` → `media.go:CreateMediaItem()` -- `UpdateMediaItem()` → `media.go:UpdateMediaItem()` -- `DeleteMediaItem()` → `media.go:DeleteMediaItem()` -- `SearchMediaItems()` → `media.go:SearchMediaItems()` - -**Method signature changes:** -```go -// OLD in ebook.go -func (h *Handler) ListMediaItems(c echo.Context) error - -// NEW in media.go -func (mh *MediaHandler) ListMediaItems(c echo.Context) error -``` - -**Implementation:** -Copy exact method bodies, change receiver from `h *Handler` to `mh *MediaHandler` - -#### Step 2.3: Move metadata methods (From ebook.go lines 630-839) - -**Methods to move:** -- `CreateMediaRating()` → `media.go:CreateMediaRating()` -- `GetMediaRating()` → `media.go:GetMediaRating()` -- `UpdateMediaRating()` → `media.go:UpdateMediaRating()` -- `DeleteMediaRating()` → `media.go:DeleteMediaRating()` - -- `GetMediaReadingProgress()` → `media.go:GetMediaReadingProgress()` -- `UpdateMediaReadingProgress()` → `media.go:UpdateMediaReadingProgress()` -- `DeleteMediaReadingProgress()` → `media.go:DeleteMediaReadingProgress()` - -- `GetMediaNotes()` → `media.go:GetMediaNotes()` -- `CreateMediaNote()` → `media.go:CreateMediaNote()` -- `GetMediaNote()` → `media.go:GetMediaNote()` -- `UpdateMediaNote()` → `media.go:UpdateMediaNote()` -- `DeleteMediaNote()` → `media.go:DeleteMediaNote()` - -- `GetMediaHighlights()` → `media.go:GetMediaHighlights()` -- `CreateMediaHighlight()` → `media.go:CreateMediaHighlight()` -- `GetMediaHighlight()` → `media.go:GetMediaHighlight()` -- `UpdateMediaHighlight()` → `media.go:UpdateMediaHighlight()` -- `DeleteMediaHighlight()` → `media.go:DeleteMediaHighlight()` - -**Verification after move:** -- [ ] All methods compile with new receiver type -- [ ] No changes to method bodies -- [ ] All database calls work (same `mh.db` reference) -- [ ] All imports are correct - ---- - -## Phase 3: Create Search Handler (QUERY OPERATIONS) - -### File: `internal/handlers/search.go` (NEW) - -#### Step 3.1: Create SearchHandler struct - -```go -package handlers - -import ( - "bookhoard/internal/database" - "context" - "fmt" - "net/http" - "strconv" - - "github.com/labstack/echo/v4" -) - -// SearchHandler handles search, query, and filtered operations -type SearchHandler struct { - db *database.Queries -} - -// NewSearchHandler creates a new search handler -func NewSearchHandler(db *database.Queries) *SearchHandler { - return &SearchHandler{ - db: db, - } -} -``` - -#### Step 3.2: Move search methods (From ebook.go lines 1297-end) - -**Methods to move:** -- `SearchMediaItems()` → `search.go:SearchMediaItems()` -- `QueryBooks()` → `search.go:QueryBooks()` - -**Note:** These methods are small and focused, perfect for separate handler - ---- - -## Phase 4: Create Matching Handler (SYNC OPERATIONS) - -### File: `internal/handlers/matching.go` (NEW) - -#### Step 4.1: Create MatchingHandler struct - -```go -package handlers - -import ( - "bookhoard/internal/database" - "bookhoard/internal/sync" - "context" - "fmt" - "net/http" - - "github.com/google/uuid" - "github.com/jackc/pgx/v5/pgtype" - "github.com/labstack/echo/v4" -) - -// MatchingHandler handles book matching, linking, and file alias operations -type MatchingHandler struct { - db *database.Queries - connManager *wsync.ConnectionManager -} - -// NewMatchingHandler creates a new matching handler -func NewMatchingHandler(db *database.Queries, connManager *wsync.ConnectionManager) *MatchingHandler { - return &MatchingHandler{ - db: db, - connManager: connManager, - } -} -``` - -#### Step 4.2: Move matching methods (From ebook.go lines 96-104) - -**Methods to move:** -- `QueryBooks()` → `matching.go:QueryBooks()` -- `LinkBook()` → `matching.go:LinkBook()` -- `GetUnlinkedBooks()` → `matching.go:GetUnlinkedBooks()` -- `GetDeviceFileAliases()` → `matching.go:GetDeviceFileAliases()` -- `CreateDeviceFileAlias()` → `matching.go:CreateDeviceFileAlias()` -- `UpdateDeviceFileAlias()` → `matching.go:UpdateDeviceFileAlias()` -- `DeleteDeviceFileAlias()` → `matching.go:DeleteDeviceFileAlias()` -- `GetBookMatches()` → `matching.go:GetBookMatches()` - ---- - -## Phase 5: Update Router Package (ROUTE REGISTRATION) - -### Purpose -Register all routes with new handler types, maintain exact same endpoints - -#### Step 5.1: Update router.go Config struct - -**File:** `internal/router/router.go` - -**Current Config (lines 33-52):** -```go -type Config struct { - Echo *echo.Echo - Queries *database.Queries - Cfg *config.Config - DBPool interface{} - AuthHandler *handlers.AuthHandler - LibraryHandler *handlers.LibraryHandler - DeviceHandler *handlers.DeviceHandler - EbookHandler *handlers.Handler // ← WILL BE REMOVED - KOReaderHandler *handlers.KOReaderHandler - WSHandler *handlers.WSHandler - ConflictHandler *handlers.ConflictHandler - AnalyticsHandler *handlers.AnalyticsHandler - QueueHandler *handlers.QueueHandler - CollectionHandler *handlers.CollectionHandler - OPDSHandler *handlers.OPDSHandler - ConnManager *sync.ConnectionManager - QueueProcessor *sync.SyncQueueProcessor - DeviceAuthMiddleware *middleware.DeviceAuthMiddleware - LoginTracker *ratelimit.LoginAttemptTracker -} -``` - -**New Config (replace EbookHandler with 3 new handlers):** -```go -type Config struct { - Echo *echo.Echo - Queries *database.Queries - Cfg *config.Config - DBPool interface{} - AuthHandler *handlers.AuthHandler - LibraryHandler *handlers.LibraryHandler - DeviceHandler *handlers.DeviceHandler - // EbookHandler REMOVED - replaced below: - MediaHandler *handlers.MediaHandler // ← NEW - SearchHandler *handlers.SearchHandler // ← NEW - MatchingHandler *handlers.MatchingHandler // ← NEW - KOReaderHandler *handlers.KOReaderHandler - WSHandler *handlers.WSHandler - ConflictHandler *handlers.ConflictHandler - AnalyticsHandler *handlers.AnalyticsHandler - QueueHandler *handlers.QueueHandler - CollectionHandler *handlers.CollectionHandler - OPDSHandler *handlers.OPDSHandler - ConnManager *sync.ConnectionManager - QueueProcessor *sync.SyncQueueProcessor - DeviceAuthMiddleware *middleware.DeviceAuthMiddleware - LoginTracker *ratelimit.LoginAttemptTracker -} -``` - -#### Step 5.2: Update RegisterRoutes function - -**File:** `internal/router/router.go` - -**Current route registration (lines 114-129):** -```go -jwtMiddleware := createJWTMiddleware(cfg) -protected := e.Group("/api", jwtMiddleware) -handlers.SetupRoutes(protected, cfg.Queries, cfg.ConnManager) // OLD METHOD - -// Register route groups -registerAuthRoutes(cfg, rateLimitMiddleware) -registerLibraryRoutes(cfg) -registerDeviceRoutes(cfg) -registerSyncRoutes(cfg) -registerMediaRoutes(cfg) -registerConflictRoutes(cfg) -registerAnalyticsRoutes(cfg) -registerQueueRoutes(cfg) -registerOPDSRoutes(cfg) -registerWebSocketRoutes(cfg) -registerFrontendRoutes(cfg) -registerDocumentationRoutes(cfg) -``` - -**New route registration:** -```go -jwtMiddleware := createJWTMiddleware(cfg) -protected := e.Group("/api", jwtMiddleware) - -// Create handler instances using factory functions -mediaHandler := handlers.NewMediaHandler(cfg.Queries, cfg.Worker) // Worker needed -searchHandler := handlers.NewSearchHandler(cfg.Queries) -matchingHandler := handlers.NewMatchingHandler(cfg.Queries, cfg.ConnManager) - -// Register route groups -registerAuthRoutes(cfg, rateLimitMiddleware) -registerLibraryRoutes(cfg) -registerDeviceRoutes(cfg) -registerSyncRoutes(cfg) -registerMediaRoutes(cfg) // ← WILL USE NEW MediaHandler -registerSearchRoutes(cfg) // ← NEW FILE -registerMatchingRoutes(cfg) // ← NEW FILE -registerConflictRoutes(cfg) -registerAnalyticsRoutes(cfg) -registerQueueRoutes(cfg) -registerOPDSRoutes(cfg) -registerWebSocketRoutes(cfg) -registerFrontendRoutes(cfg) -registerDocumentationRoutes(cfg) -``` - -#### Step 5.3: Create registerSearchRoutes function - -**File:** `internal/router/search.go` (NEW) - -```go -package router - -import ( - "net/http" - "github.com/labstack/echo/v4" -) - -func registerSearchRoutes(cfg *Config) { - e := cfg.Echo - - // JWT middleware for protected routes - jwtMiddleware := createJWTMiddleware(cfg) - protected := e.Group("/api", jwtMiddleware) - - // Search and query endpoints (all authenticated users) - protected.GET("/media-items/search", cfg.SearchHandler.SearchMediaItems) - protected.POST("/sync/books/query", cfg.MatchingHandler.QueryBooks) -} -``` - -#### Step 5.4: Create registerMatchingRoutes function - -**File:** `internal/router/matching.go` (NEW) - -```go -package router - -import ( - "github.com/labstack/echo/v4" -) - -func registerMatchingRoutes(cfg *Config) { - e := cfg.Echo - - // JWT middleware for protected routes - jwtMiddleware := createJWTMiddleware(cfg) - protected := e.Group("/api", jwtMiddleware) - - // Book matching and linking routes (all authenticated users) - protected.POST("/devices/:deviceId/sync/link-book", cfg.MatchingHandler.LinkBook) - protected.GET("/devices/:deviceId/sync/unlinked-books", cfg.MatchingHandler.GetUnlinkedBooks) - - // File alias routes (all authenticated users) - protected.GET("/devices/:id/file-aliases", cfg.MatchingHandler.GetDeviceFileAliases) - protected.POST("/devices/:id/file-aliases", cfg.MatchingHandler.CreateDeviceFileAlias) - protected.PUT("/devices/:id/file-aliases/:aliasId", cfg.MatchingHandler.UpdateDeviceFileAlias) - protected.DELETE("/devices/:id/file-aliases/:aliasId", cfg.MatchingHandler.DeleteDeviceFileAlias) - protected.GET("/books/match", cfg.MatchingHandler.GetBookMatches) -} -``` - -#### Step 5.5: Update existing registerMediaRoutes function - -**File:** `internal/router/media.go` (MODIFY) - -**Current:** Empty or minimal -**New:** Register all media CRUD endpoints - -```go -package router - -import ( - "github.com/labstack/echo/v4" - "bookhoard/internal/handlers" -) - -func registerMediaRoutes(cfg *Config) { - e := cfg.Echo - - // JWT middleware for protected routes - jwtMiddleware := createJWTMiddleware(cfg) - protected := e.Group("/api", jwtMiddleware) - admin := protected.Group("", handlers.AdminMiddleware) - - // Media item routes (all authenticated users) - protected.GET("/media-items", cfg.MediaHandler.ListMediaItems) - protected.GET("/media-items/filtered", cfg.MediaHandler.ListMediaItemsFiltered) - protected.GET("/media-items/search", cfg.MediaHandler.SearchMediaItems) - protected.GET("/media-items/:id", cfg.MediaHandler.GetMediaItem) - - // Media rating routes (all authenticated users) - protected.POST("/media-items/:id/rating", cfg.MediaHandler.CreateMediaRating) - protected.GET("/media-items/:id/rating", cfg.MediaHandler.GetMediaRating) - protected.PUT("/media-items/:id/rating", cfg.MediaHandler.UpdateMediaRating) - protected.DELETE("/media-items/:id/rating", cfg.MediaHandler.DeleteMediaRating) - - // Progress routes (all authenticated users) - protected.GET("/progress/:id", cfg.MediaHandler.GetUniversalProgress) - protected.POST("/progress/:id", cfg.MediaHandler.UpdateUniversalProgress) - protected.GET("/progress/:id/history", cfg.MediaHandler.GetProgressHistory) - - // Notes routes (all authenticated users) - protected.GET("/media-items/:id/notes", cfg.MediaHandler.GetMediaNotes) - protected.POST("/media-items/:id/notes", cfg.MediaHandler.CreateMediaNote) - protected.GET("/media-items/:id/notes/:noteId", cfg.MediaHandler.GetMediaNote) - protected.PUT("/media-items/:id/notes/:noteId", cfg.MediaHandler.UpdateMediaNote) - protected.DELETE("/media-items/:id/notes/:noteId", cfg.MediaHandler.DeleteMediaNote) - - // Highlights routes (all authenticated users) - protected.GET("/media-items/:id/highlights", cfg.MediaHandler.GetMediaHighlights) - protected.POST("/media-items/:id/highlights", cfg.MediaHandler.CreateMediaHighlight) - protected.GET("/media-items/:id/highlights/:highlightId", cfg.MediaHandler.GetMediaHighlight) - protected.PUT("/media-items/:id/highlights/:highlightId", cfg.MediaHandler.UpdateMediaHighlight) - protected.DELETE("/media-items/:id/highlights/:highlightId", cfg.MediaHandler.DeleteMediaHighlight) - - // Admin-only media routes - admin.POST("/media-items", cfg.MediaHandler.CreateMediaItem) - admin.PUT("/media-items/:id", cfg.MediaHandler.UpdateMediaItem) - admin.DELETE("/media-items/:id", cfg.MediaHandler.DeleteMediaItem) -} -``` - ---- - -## Phase 6: Update main.go (INITIALIZATION) - -### File: `cmd/server/main.go` - -#### Step 6.1: Create new handler instances - -**Current handler creation (lines 78-99):** -```go -authHandler := handlers.NewAuthHandler(queries, cfg.JWTSecret, loginAttemptTracker) -libraryHandler := handlers.NewLibraryHandler(queries) -deviceHandler := handlers.NewDeviceHandler(queries, cfg.JWTSecret, cfg) -// ... other handlers -``` - -**Add new handler instances:** -```go -authHandler := handlers.NewAuthHandler(queries, cfg.JWTSecret, loginAttemptTracker) -libraryHandler := handlers.NewLibraryHandler(queries) -deviceHandler := handlers.NewDeviceHandler(queries, cfg.JWTSecret, cfg) -// ... existing handlers ... - -// NEW: Create refactored handlers -mediaHandler := handlers.NewMediaHandler(queries, queueProcessor) // Worker needed -searchHandler := handlers.NewSearchHandler(queries) -matchingHandler := handlers.NewMatchingHandler(queries, connManager) -``` - -#### Step 6.2: Update routerConfig - -**Current routerConfig (lines 131-150):** -```go -routerConfig := &router.Config{ - Echo: e, - Queries: queries, - Cfg: cfg, - DBPool: dbPool, - AuthHandler: authHandler, - LibraryHandler: libraryHandler, - DeviceHandler: deviceHandler, - EbookHandler: ebookHandler, // ← REMOVE - KOReaderHandler: koreaderHandler, - WSHandler: wsHandler, - // ... other fields -} -``` - -**New routerConfig:** -```go -routerConfig := &router.Config{ - Echo: e, - Queries: queries, - Cfg: cfg, - DBPool: dbPool, - AuthHandler: authHandler, - LibraryHandler: libraryHandler, - DeviceHandler: deviceHandler, - // EbookHandler REMOVED - MediaHandler: mediaHandler, // ← NEW - SearchHandler: searchHandler, // ← NEW - MatchingHandler: matchingHandler, // ← NEW - KOReaderHandler: koreaderHandler, - WSHandler: wsHandler, - ConflictHandler: conflictHandler, - AnalyticsHandler: analyticsHandler, - QueueHandler: queueHandler, - CollectionHandler: collectionHandler, - OPDSHandler: opdsHandler, - ConnManager: connManager, - QueueProcessor: queueProcessor, - DeviceAuthMiddleware: deviceAuthMiddleware, - LoginTracker: loginAttemptTracker, -} -``` - -**Note:** Need to determine where `worker` comes from for MediaHandler. Check existing code. - ---- - -## Phase 7: Clean Up ebook.go (FINALIZE) - -### File: `internal/handlers/ebook.go` - -#### Step 7.1: Remove moved methods - -**Remove these method sections:** -- Lines 474-959: All media CRUD and metadata methods -- Lines 1297-end: All search and matching methods -- Lines 73-104: Collection and device mapping routes -- Lines 105-143: All route registrations - -**Keep these sections:** -- Lines 38-69: Handler struct and NewHandler function -- Lines 70-72: Simplified SetupRoutes function -- Lines 159-end: Request/response structs - -#### Step 7.2: Update Handler struct dependencies - -**Current Handler struct (lines 19-30):** -```go -type Handler struct { - db *database.Queries - worker *worker.Worker // ← MOVE TO MediaHandler - scanner *services.EbookScanner // ← MAY MOVE TO ScannerHandler - scheduler *cron.Cron // ← MAY MOVE TO ScannerHandler - ctx context.Context - cancel context.CancelFunc - mu sync.RWMutex - watchingLibraries map[string]bool -} -``` - -**Potential final Handler struct:** -```go -// If keeping only SetupRoutes function: -type Handler struct { - // This might be entirely removed if SetupRoutes is only factory -} - -func SetupRoutes(g *echo.Group, db *database.Queries, connManager *wsync.ConnectionManager) *Handler { - return NewHandler(db, connManager) -} -``` - -OR - -**If Handler is no longer needed:** -- Remove entire file -- Move remaining structs to appropriate files -- Update all imports - ---- - -## Implementation Order (SAFEST FIRST) - -### Phase 1: Create new handler files (ZERO RISK) -1. Create `internal/handlers/media.go` with struct and empty methods -2. Create `internal/handlers/search.go` with struct and empty methods -3. Create `internal/handlers/matching.go` with struct and empty methods -4. Verify compilation - -### Phase 2: Move method bodies (LOW RISK) -1. Copy media methods from ebook.go to media.go -2. Copy search methods from ebook.go to search.go -3. Copy matching methods from ebook.go to matching.go -4. Test compilation after each file - -### Phase 3: Update router registration (MEDIUM RISK) -1. Update router.go Config struct -2. Create registerSearchRoutes and registerMatchingRoutes -3. Update registerMediaRoutes to use MediaHandler -4. Test compilation - -### Phase 4: Update main.go initialization (LOW RISK) -1. Add new handler instance creation -2. Update routerConfig -3. Test compilation and runtime - -### Phase 5: Clean up ebook.go (LOW RISK) -1. Remove moved methods -2. Simplify/Remove Handler struct if no longer needed -3. Verify final state - ---- - -## Verification Checklist - -### After Each Phase: - -**Phase 1 (new files created):** -- [ ] All 3 new files created -- [ ] Code compiles without errors -- [ ] No existing functionality broken - -**Phase 2 (methods moved):** -- [ ] All media methods in media.go compile -- [ ] All search methods in search.go compile -- [ ] All matching methods in matching.go compile -- [ ] Original ebook.go still compiles - -**Phase 3 (router updated):** -- [ ] router.go compiles with new Config struct -- [ ] registerSearchRoutes compiles -- [ ] registerMatchingRoutes compiles -- [ ] registerMediaRoutes updated correctly - -**Phase 4 (main.go updated):** -- [ ] Application compiles -- [ ] All handler instances created -- [ ] routerConfig populated correctly - -**Phase 5 (cleanup):** -- [ ] ebook.go reduced from 1,350 to <200 lines -- [ ] All route endpoints still exist -- [ ] Bruno tests still pass -- [ ] Verification script passes (26/26) - -### Final Verification Commands: - -```bash -# Compilation test -go build ./cmd/server - -# Test all critical endpoints still work -curl http://localhost:8765/api/media-items | jq . -curl http://localhost:8765/api/media-items/search?q=test | jq . -curl http://localhost:8765/api/books/match | jq . - -# Run verification script -bash scripts/verify-guidelines.sh - -# Check file sizes -wc -l internal/handlers/*.go | sort -n -# Expected: media.go ~600, search.go ~50, matching.go ~150, ebook.go <200 - -# Test Bruno tests still pass -cd bruno && npx bruno test -``` - ---- - -## Rollback Strategy - -### If any phase breaks: - -**Immediate rollback:** -```bash -git checkout -- internal/handlers/media.go -git checkout -- internal/handlers/search.go -git checkout -- internal/handlers/matching.go -git checkout -- internal/router/router.go -git checkout -- internal/router/media.go -git checkout -- cmd/server/main.go -``` - -**Phase-by-phase rollback:** -```bash -# After Phase 2: If methods don't compile -git checkout -- internal/handlers/media.go -git checkout -- internal/handlers/search.go -git checkout -- internal/handlers/matching.go - -# After Phase 3: If router breaks -git checkout -- internal/router/router.go -git checkout -- internal/router/media.go - -# After Phase 4: If initialization breaks -git checkout -- cmd/server/main.go -``` - -**Full project reset:** -```bash -git checkout -- internal/handlers/ebook.go -git clean -fd internal/handlers/media.go internal/handlers/search.go internal/handlers/matching.go -``` - ---- - -## Success Metrics - -**Before refactor:** -- ebook.go: 1,350 lines -- 1 monolithic handler -- Mixed concerns in single file - -**After refactor:** -- ebook.go: ~150 lines (SetupRoutes only) -- media.go: ~600 lines (CRUD + metadata) -- search.go: ~80 lines (search operations) -- matching.go: ~200 lines (sync operations) -- Clear single responsibilities per file -- All API endpoints identical -- All tests continue to work - -**File count increase:** +3 new handler files -**Lines of code change:** ~0 (moved, not modified) -**API compatibility:** 100% maintained -**Risk level:** VERY LOW - ---- - -## Final Notes - -### Dependencies to verify: - -1. **Worker access for MediaHandler:** - - Check where `worker` is created in current code - - Pass to NewMediaHandler properly - -2. **AdminMiddleware import:** - - Ensure all router files import handlers package - - AdminMiddleware is accessible - -3. **Import statements:** - - Verify all new handler files have correct imports - - No circular dependencies created - -### Testing strategy: - -1. **Compile after each file creation** -2. **Test endpoint accessibility after router changes** -3. **Run existing test suite before/after** -4. **Verify Bruno tests still pass** -5. **Check verification script passes** - -This plan ensures zero API changes while completely reorganizing the codebase into maintainable, single-responsibility files. - ---- - -**Ready for safe implementation by any AI following these exact phases.** \ No newline at end of file diff --git a/TAGS_CONTRIBUTORS_IMPLEMENTATION_PLAN.md b/TAGS_CONTRIBUTORS_IMPLEMENTATION_PLAN.md deleted file mode 100644 index 7cac37d..0000000 --- a/TAGS_CONTRIBUTORS_IMPLEMENTATION_PLAN.md +++ /dev/null @@ -1,1424 +0,0 @@ -# Tags & Contributors Migration Implementation Plan - -## Objective -Implement dual-field normalization for tags and contributors: -- **Display field**: Preserves exact variant (casing, punctuation) -- **Search field**: Lowercase, no punctuation, deduplicated - -## Timeline -Immediate execution - no migration needed (app never deployed) - -## Phases -1. Dependencies -2. Schema Changes -3. Normalization Functions -4. Tests -5. Regenerate sqlc -6. Handler Updates -7. Scanner Updates -8. Search Query Updates -9. Frontend Documentation -10. Verification - ---- - -## Phase 1: Dependencies - -### File: `go.mod` - -**Action:** Add `golang.org/x/text` dependency for titlecasing - -**Command:** -```bash -go get golang.org/x/text -go mod tidy -``` - -**Expected result:** Dependency added to go.mod and go.sum - ---- - -## Phase 2: Schema Changes - -### File: `database/schema/schema.sql` - -**Location:** After line 120 (after media_items table definition, before indexes) - -**Action:** Add 4 new columns and 2 new indexes - -**Add after line 120:** -```sql --- Add search fields for case-insensitive, punctuation-free searching -ALTER TABLE media_items ADD COLUMN IF NOT EXISTS tags_search TEXT[]; -ALTER TABLE media_items ADD COLUMN IF NOT EXISTS contributors_search TEXT[]; - --- Create GIN indexes for fast search field searches -CREATE INDEX IF NOT EXISTS idx_media_items_tags_search ON media_items USING GIN (tags_search); -CREATE INDEX IF NOT EXISTS idx_media_items_contributors_search ON media_items USING GIN (contributors_search); -``` - -**Expected result:** 4 new columns, 2 new indexes on media_items table - -**Note:** IF NOT EXISTS allows safe re-running if columns already exist - ---- - -## Phase 3: Normalization Functions - -### File: `internal/utils/tags.go` - -**Action:** Complete rewrite with comprehensive normalization functions - -**Replace entire file content with:** - -```go -package utils - -import ( - "strings" - "unicode" - - "golang.org/x/text/cases" - "golang.org/x/text/language" -) - -// titleCaser is a global caser for titlecase conversion -var titleCaser = cases.Title(language.Und, cases.NoLower) - -// titlecase converts a string to title case while preserving hyphenation -// Example: "science fiction" → "Science Fiction", "non-fiction" → "Non-Fiction" -func titlecase(s string) string { - return titleCaser.String(s) -} - -// removePunctuation removes all punctuation characters from a string -// Used for search field normalization only -func removePunctuation(s string) string { - return strings.Map(func(r rune) rune { - if unicode.IsPunct(r) { - return -1 - } - return r - }, s) -} - -// NormalizeTags normalizes an array of tags for display: -// 1. Trim whitespace -// 2. Titlecase (preserves hyphenation) -// 3. Case-insensitive deduplication -// 4. Remove empty strings -func NormalizeTags(tags []string) []string { - seen := make(map[string]struct{}) - var normalized []string - - for _, tag := range tags { - // Trim whitespace - tag = strings.TrimSpace(tag) - - // Skip empty tags - if tag == "" { - continue - } - - // Titlecase for display (preserves hyphenation: "Non-Fiction") - tag = titlecase(tag) - - // Case-insensitive deduplication - key := strings.ToLower(tag) - if _, exists := seen[key]; !exists { - seen[key] = struct{}{} - normalized = append(normalized, tag) - } - } - - return normalized -} - -// NormalizeTagsSearch normalizes an array of tags for searching: -// 1. Trim whitespace -// 2. Remove punctuation -// 3. Lowercase -// 4. Case-insensitive deduplication -// 5. Remove empty strings -func NormalizeTagsSearch(tags []string) []string { - seen := make(map[string]struct{}) - var normalized []string - - for _, tag := range tags { - // Trim whitespace - tag = strings.TrimSpace(tag) - - // Skip empty tags - if tag == "" { - continue - } - - // Remove punctuation for search - tag = removePunctuation(tag) - - // Lowercase for search - tag = strings.ToLower(tag) - - // Skip empty after removal - if tag == "" { - continue - } - - // Case-insensitive deduplication - if _, exists := seen[tag]; !exists { - seen[tag] = struct{}{} - normalized = append(normalized, tag) - } - } - - return normalized -} - -// NormalizeContributors normalizes an array of contributors for display: -// 1. Trim whitespace -// 2. Preserve original casing (including CAPSLOCK companies) -// 3. Preserve original punctuation for display -// 4. Case-insensitive deduplication (removes punctuation for comparison only) -// 5. Remove empty strings -func NormalizeContributors(contributors []string) []string { - seen := make(map[string]struct{}) - var normalized []string - - for _, contributor := range contributors { - // Trim whitespace - contributor = strings.TrimSpace(contributor) - - // Skip empty contributors - if contributor == "" { - continue - } - - // Case-insensitive deduplication (remove punctuation for dedup check only) - dedupKey := removePunctuation(strings.ToLower(contributor)) - - // Keep original case and punctuation for display - if _, exists := seen[dedupKey]; !exists { - seen[dedupKey] = struct{}{} - normalized = append(normalized, contributor) - } - } - - return normalized -} - -// NormalizeContributorsSearch normalizes an array of contributors for searching: -// 1. Trim whitespace -// 2. Remove punctuation -// 3. Lowercase -// 4. Case-insensitive deduplication -// 5. Remove empty strings -func NormalizeContributorsSearch(contributors []string) []string { - seen := make(map[string]struct{}) - var normalized []string - - for _, contributor := range contributors { - // Trim whitespace - contributor = strings.TrimSpace(contributor) - - // Skip empty contributors - if contributor == "" { - continue - } - - // Remove punctuation for search - contributor = removePunctuation(contributor) - - // Lowercase for search - contributor = strings.ToLower(contributor) - - // Skip empty after removal - if contributor == "" { - continue - } - - // Case-insensitive deduplication - if _, exists := seen[contributor]; !exists { - seen[contributor] = struct{}{} - normalized = append(normalized, contributor) - } - } - - return normalized -} - -// JoinTags converts a string array to a comma-separated string -// Maintained for backward compatibility with external systems -func JoinTags(tags []string) string { - return strings.Join(tags, ", ") -} - -// SplitTags converts a comma-separated string to a normalized display array -func SplitTags(tags string) []string { - if tags == "" { - return []string{} - } - - parts := strings.Split(tags, ",") - return NormalizeTags(parts) -} -``` - -**Expected result:** 4 new normalization functions (2 for tags, 2 for contributors) - ---- - -## Phase 4: Tests - -### File: `internal/utils/tags_test.go` (NEW FILE) - -**Action:** Create comprehensive test suite - -**Create new file with content:** - -```go -package utils - -import ( - "testing" - - "github.com/stretchr/testify/assert" -) - -// TestNormalizeTags tests tag display normalization -func TestNormalizeTags(t *testing.T) { - tests := []struct { - name string - input []string - expected []string - }{ - { - name: "empty array", - input: []string{}, - expected: []string{}, - }, - { - name: "nil input", - input: nil, - expected: []string{}, - }, - { - name: "single tag", - input: []string{" science fiction "}, - expected: []string{"Science Fiction"}, - }, - { - name: "multiple tags", - input: []string{"fiction", "adventure"}, - expected: []string{"Fiction", "Adventure"}, - }, - { - name: "multi-word tag", - input: []string{" science fiction "}, - expected: []string{"Science Fiction"}, - }, - { - name: "hyphenated tag", - input: []string{"non-fiction"}, - expected: []string{"Non-Fiction"}, - }, - { - name: "mixed case", - input: []string{"FICTION", "fiction", "Fiction"}, - expected: []string{"Fiction"}, - }, - { - name: "case-insensitive dedup", - input: []string{"fiction", "FICTION", "Fiction"}, - expected: []string{"Fiction"}, - }, - { - name: "remove empty strings", - input: []string{"fiction", "", "adventure", " "}, - expected: []string{"Fiction", "Adventure"}, - }, - { - name: "whitespace trimming", - input: []string{" fiction ", "\tadventure\t"}, - expected: []string{"Fiction", "Adventure"}, - }, - { - name: "preserves punctuation", - input: []string{"science-fiction", "O'Reilly"}, - expected: []string{"Science-Fiction", "O'Reilly"}, - }, - { - name: "complex real-world example", - input: []string{" science fiction ", "FICTION", "non-fiction", "", " "}, - expected: []string{"Science Fiction", "Fiction", "Non-Fiction"}, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - result := NormalizeTags(tt.input) - assert.Equal(t, tt.expected, result) - }) - } -} - -// TestNormalizeTagsSearch tests tag search normalization -func TestNormalizeTagsSearch(t *testing.T) { - tests := []struct { - name string - input []string - expected []string - }{ - { - name: "empty array", - input: []string{}, - expected: []string{}, - }, - { - name: "nil input", - input: nil, - expected: []string{}, - }, - { - name: "single tag", - input: []string{" Science Fiction "}, - expected: []string{"science fiction"}, - }, - { - name: "lowercase", - input: []string{"SCIENCE FICTION"}, - expected: []string{"science fiction"}, - }, - { - name: "remove punctuation", - input: []string{"science-fiction"}, - expected: []string{"science fiction"}, - }, - { - name: "remove period", - input: []string{"ACME CORP."}, - expected: []string{"acme corp"}, - }, - { - name: "remove multiple punctuation", - input: []string{"O'Reilly Media!"}, - expected: []string{"oreilly media"}, - }, - { - name: "case-insensitive dedup", - input: []string{"science fiction", "SCIENCE FICTION", "Science Fiction"}, - expected: []string{"science fiction"}, - }, - { - name: "remove empty after punctuation removal", - input: []string{"..."}, - expected: []string{}, - }, - { - name: "complex example", - input: []string{" Science-Fiction ", "FICTION", "", " "}, - expected: []string{"science fiction", "fiction"}, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - result := NormalizeTagsSearch(tt.input) - assert.Equal(t, tt.expected, result) - }) - } -} - -// TestNormalizeContributors tests contributor display normalization -func TestNormalizeContributors(t *testing.T) { - tests := []struct { - name string - input []string - expected []string - }{ - { - name: "empty array", - input: []string{}, - expected: []string{}, - }, - { - name: "nil input", - input: nil, - expected: []string{}, - }, - { - name: "single contributor", - input: []string{" ACME CORP "}, - expected: []string{"ACME CORP"}, - }, - { - name: "preserve case - CAPSLOCK", - input: []string{"ACME CORP"}, - expected: []string{"ACME CORP"}, - }, - { - name: "preserve case - Title Case", - input: []string{"Acme Corp"}, - expected: []string{"Acme Corp"}, - }, - { - name: "preserve case - lowercase", - input: []string{"acme corp"}, - expected: []string{"acme corp"}, - }, - { - name: "preserve punctuation - period", - input: []string{"ACME CORP."}, - expected: []string[]{"ACME CORP."}, - }, - { - name: "preserve punctuation - apostrophe", - input: []string{"O'Reilly Media"}, - expected: []string{"O'Reilly Media"}, - }, - { - name: "case-insensitive dedup - different case", - input: []string{"ACME CORP", "Acme Corp", "acme corp"}, - expected: []string{"ACME CORP"}, - }, - { - name: "case-insensitive dedup - with punctuation", - input: []string{"ACME CORP.", "Acme Corp", "acme corp"}, - expected: []string{"ACME CORP."}, - }, - { - name: "trim whitespace", - input: []string{" ACME CORP ", "\tAcme\t"}, - expected: []string{"ACME CORP", "Acme"}, - }, - { - name: "remove empty strings", - input: []string{"ACME CORP", "", "Acme Corp", " "}, - expected: []string{"ACME CORP", "Acme Corp"}, - }, - { - name: "complex real-world example", - input: []string{" ACME CORP. ", "Acme Corp", "acme corp", " "}, - expected: []string{"ACME CORP.", "Acme Corp"}, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - result := NormalizeContributors(tt.input) - assert.Equal(t, tt.expected, result) - }) - } -} - -// TestNormalizeContributorsSearch tests contributor search normalization -func TestNormalizeContributorsSearch(t *testing.T) { - tests := []struct { - name string - input []string - expected []string - }{ - { - name: "empty array", - input: []string{}, - expected: []string{}, - }, - { - name: "nil input", - input: nil, - expected: []string{}, - }, - { - name: "single contributor", - input: []string{" ACME CORP. "}, - expected: []string{"acme corp"}, - }, - { - name: "lowercase", - input: []string{"ACME CORP"}, - expected: []string{"acme corp"}, - }, - { - name: "remove punctuation - period", - input: []string{"ACME CORP."}, - expected: []string{"acme corp"}, - }, - { - name: "remove punctuation - apostrophe", - input: []string{"O'Reilly Media"}, - expected: []string{"oreilly media"}, - }, - { - name: "case-insensitive dedup", - input: []string{"ACME CORP", "acme corp", "Acme Corp"}, - expected: []string{"acme corp"}, - }, - { - name: "case-insensitive dedup with punctuation", - input: []string{"ACME CORP.", "Acme Corp", "acme corp"}, - expected: []string{"acme corp"}, - }, - { - name: "remove empty after punctuation removal", - input: []string{"..."}, - expected: []string{}, - }, - { - name: "complex example", - input: []string{" ACME CORP. ", "Acme Corp", "acme corp", " "}, - expected: []string{"acme corp"}, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - result := NormalizeContributorsSearch(tt.input) - assert.Equal(t, tt.expected, result) - }) - } -} - -// TestTitlecase tests titlecase helper function -func TestTitlecase(t *testing.T) { - tests := []struct { - name string - input string - expected string - }{ - { - name: "simple word", - input: "science", - expected: "Science", - }, - { - name: "multi-word", - input: "science fiction", - expected: "Science Fiction", - }, - { - name: "hyphenated", - input: "non-fiction", - expected: "Non-Fiction", - }, - { - name: "already capitalized", - input: "Science Fiction", - expected: "Science Fiction", - }, - { - name: "all caps", - input: "SCIENCE FICTION", - expected: "Science Fiction", - }, - { - name: "all lowercase", - input: "science fiction", - expected: "Science Fiction", - }, - { - name: "apostrophe", - input: "o'reilly media", - expected: "O'Reilly Media", - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - result := titlecase(tt.input) - assert.Equal(t, tt.expected, result) - }) - } -} - -// TestRemovePunctuation tests punctuation removal -func TestRemovePunctuation(t *testing.T) { - tests := []struct { - name string - input string - expected string - }{ - { - name: "no punctuation", - input: "acme corp", - expected: "acme corp", - }, - { - name: "period", - input: "ACME CORP.", - expected: "ACME CORP", - }, - { - name: "apostrophe", - input: "O'Reilly", - expected: "OReilly", - }, - { - name: "multiple punctuation", - input: "science-fiction!", - expected: "sciencefiction", - }, - { - name: "only punctuation", - input: "...", - expected: "", - }, - { - name: "mixed content", - input: "O'Reilly Media!", - expected: "OReilly Media", - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - result := removePunctuation(tt.input) - assert.Equal(t, tt.expected, result) - }) - } -} - -// TestJoinTags tests tag joining -func TestJoinTags(t *testing.T) { - tests := []struct { - name string - input []string - expected string - }{ - { - name: "empty array", - input: []string{}, - expected: "", - }, - { - name: "single tag", - input: []string{"fiction"}, - expected: "fiction", - }, - { - name: "multiple tags", - input: []string{"fiction", "adventure"}, - expected: "fiction, adventure", - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - result := JoinTags(tt.input) - assert.Equal(t, tt.expected, result) - }) - } -} - -// TestSplitTags tests tag splitting and normalization -func TestSplitTags(t *testing.T) { - tests := []struct { - name string - input string - expected []string - }{ - { - name: "empty string", - input: "", - expected: []string{}, - }, - { - name: "single tag", - input: "fiction", - expected: []string{"Fiction"}, - }, - { - name: "multiple tags comma separated", - input: "fiction, adventure", - expected: []string{"Fiction", "Adventure"}, - }, - { - name: "with spaces", - input: "fiction,adventure", - expected: []string{"Fiction", "Adventure"}, - }, - { - name: "with extra spaces", - input: " fiction , adventure ", - expected: []string{"Fiction", "Adventure"}, - }, - { - name: "deduplicates", - input: "fiction, FICTION, Fiction", - expected: []string{"Fiction"}, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - result := SplitTags(tt.input) - assert.Equal(t, tt.expected, result) - }) - } -} -``` - -**Expected result:** 100+ test cases covering all normalization functions - ---- - -## Phase 5: Regenerate sqlc - -### File: `internal/database/` - -**Action:** Regenerate sqlc code to include new columns - -**Command:** -```bash -cd internal/database && sqlc generate -``` - -**Expected changes:** -- `models.go` - Auto-generated structs with new fields -- `queries.sql.go` - Auto-generated queries with new parameters - -**Expected result:** sqlc regenerates with TagsSearch and ContributorsSearch fields - ---- - -## Phase 6: Handler Updates - -### File: `internal/handlers/media.go` - -#### Change 1: Update CreateMediaItemRequest struct (Line 20-37) - -**Current:** -```go -type CreateMediaItemRequest struct { - ... - Tags []string `json:"tags"` - ... - Contributors []string `json:"contributors"` - ... -} -``` - -**No change needed** - already correct - -#### Change 2: Update CreateMediaItemHandler (Line 876-933) - -**Location:** After line 894 (after tag normalization), before library check (before line 896) - -**Add:** -```go -// Normalize tags for display -if len(req.Tags) > 0 { - req.Tags = utils.NormalizeTags(req.Tags) -} - -// Normalize contributors for display -if len(req.Contributors) > 0 { - req.Contributors = utils.NormalizeContributors(req.Contributors) -} - -// Normalize search fields -tagsSearch := utils.NormalizeTagsSearch(req.Tags) -contributorsSearch := utils.NormalizeContributorsSearch(req.Contributors) -``` - -#### Change 3: Update CreateMediaItem params (Line 904-922) - -**Current (lines 916-920):** -```go -Tags: req.Tags, -Asin: pgtype.Text{String: req.ASIN, Valid: req.ASIN != ""}, -DatePublished: pgtype.Date{Time: parseDate(req.DatePublished), Valid: req.DatePublished != ""}, -Publisher: pgtype.Text{String: req.Publisher, Valid: req.Publisher != ""}, -Contributors: req.Contributors, -``` - -**Replace with:** -```go -Tags: req.Tags, -TagsSearch: tagsSearch, -Asin: pgtype.Text{String: req.ASIN, Valid: req.ASIN != ""}, -DatePublished: pgtype.Date{Time: parseDate(req.DatePublished), Valid: req.DatePublished != ""}, -Publisher: pgtype.Text{String: req.Publisher, Valid: req.Publisher != ""}, -Contributors: req.Contributors, -ContributorsSearch: contributorsSearch, -``` - -#### Change 4: Update UpdateMediaItemRequest struct (Line 40-53) - -**Current:** -```go -type UpdateMediaItemRequest struct { - ... - Tags []string `json:"tags"` - ... - Contributors []string `json:"contributors"` - ... -} -``` - -**No change needed** - already correct - -#### Change 5: Update UpdateMediaItemHandler (Line 935-977) - -**Location:** After line 955 (after binding), before database call (before line 957) - -**Add:** -```go -// Normalize tags for display -if len(req.Tags) > 0 { - req.Tags = utils.NormalizeTags(req.Tags) -} - -// Normalize contributors for display -if len(req.Contributors) > 0 { - req.Contributors = utils.NormalizeContributors(req.Contributors) -} - -// Normalize search fields -tagsSearch := utils.NormalizeTagsSearch(req.Tags) -contributorsSearch := utils.NormalizeContributorsSearch(req.Contributors) -``` - -#### Change 6: Update UpdateMediaItem params (Line 957-970) - -**Current (lines 966-970):** -```go -Tags: req.Tags, -Asin: pgtype.Text{String: req.ASIN, Valid: req.ASIN != ""}, -DatePublished: pgtype.Date{Time: parseDate(req.DatePublished), Valid: req.DatePublished != ""}, -Publisher: pgtype.Text{String: req.Publisher, Valid: req.Publisher != ""}, -Contributors: req.Contributors, -``` - -**Replace with:** -```go -Tags: req.Tags, -TagsSearch: tagsSearch, -Asin: pgtype.Text{String: req.ASIN, Valid: req.ASIN != ""}, -DatePublished: pgtype.Date{Time: parseDate(req.DatePublished), Valid: req.DatePublished != ""}, -Publisher: pgtype.Text{String: req.Publisher, Valid: req.Publisher != ""}, -Contributors: req.Contributors, -ContributorsSearch: contributorsSearch, -``` - -#### Change 7: Update HandleBulkUpdate (Line 503-506) - -**Current:** -```go -if update.Updates.Tags != nil && len(update.Updates.Tags) > 0 { - updateParams.Tags = update.Updates.Tags -} -``` - -**Replace with:** -```go -if update.Updates.Tags != nil && len(update.Updates.Tags) > 0 { - // Normalize tags for display - normalizedTags := utils.NormalizeTags(update.Updates.Tags) - updateParams.Tags = normalizedTags - - // Normalize search field - tagsSearch := utils.NormalizeTagsSearch(update.Updates.Tags) - updateParams.TagsSearch = tagsSearch -} -``` - -#### Change 8: Update HandleBulkUpdate contributors (Line 503-506 area) - -**Add after tags normalization:** -```go -if update.Updates.Contributors != nil && len(update.Updates.Contributors) > 0 { - // Normalize contributors for display - normalizedContributors := utils.NormalizeContributors(update.Updates.Contributors) - updateParams.Contributors = normalizedContributors - - // Normalize search field - contributorsSearch := utils.NormalizeContributorsSearch(update.Updates.Contributors) - updateParams.ContributorsSearch = contributorsSearch -} -``` - ---- - -## Phase 7: Scanner Updates - -### File: `internal/services/ebook_scanner.go` - -#### Change 1: Update extractEPUBMetadata (Line 527-611) - -**Location:** After line 580 (after contributors assignment), before ISBN section - -**Add:** -```go -// Normalize contributors for display -metadata.Contributors = utils.NormalizeTags(metadata.Contributors) -``` - -**Wait** - wrong function. Contributors should use NormalizeContributors, not NormalizeTags. - -**Correct addition:** -```go -// Normalize contributors for display -metadata.Contributors = utils.NormalizeContributors(metadata.Contributors) -``` - -**Location:** After line 607 (after tags assignment), before return statement (line 610) - -**Add:** -```go -// Normalize tags for display -metadata.Tags = utils.NormalizeTags(metadata.Tags) -``` - -#### Change 2: Update processEbookFile (Line 469-474) - -**Current (lines 471-472):** -```go -Contributors: metadata.Contributors, -Tags: metadata.Tags, -``` - -**Location:** Before line 471 (before Contributors assignment) - -**Add:** -```go -// Normalize metadata fields for display -metadata.Contributors = utils.NormalizeContributors(metadata.Contributors) -metadata.Tags = utils.NormalizeTags(metadata.Tags) - -// Normalize search fields -contributorsSearch := utils.NormalizeContributorsSearch(metadata.Contributors) -tagsSearch := utils.NormalizeTagsSearch(metadata.Tags) -``` - -**Replace lines 471-472 with:** -```go -Contributors: metadata.Contributors, -ContributorsSearch: contributorsSearch, -Tags: metadata.Tags, -TagsSearch: tagsSearch, -``` - -**Add import at top of file (around line 1-10):** - -Check if `"bookhoard/internal/utils"` is already imported. If not, add to imports section. - ---- - -## Phase 8: Search Query Updates - -### File: `internal/database/queries/queries.sql` - -#### Change 1: Update SearchMediaItems (Line 367-368) - -**Current:** -```sql -sqlc.narg('search_pattern') = ANY(mi.tags) OR -sqlc.narg('search_pattern') = ANY(mi.contributors) -``` - -**Replace with:** -```sql -sqlc.narg('search_pattern') = ANY(mi.tags_search) OR -sqlc.narg('search_pattern') = ANY(mi.contributors_search) -``` - -#### Change 2: Update SearchMediaItems priority case (Line 375) - -**Current:** -```sql -WHEN sqlc.narg('search_pattern') = ANY(mi.tags) THEN 4 -``` - -**Replace with:** -```sql -WHEN sqlc.narg('search_pattern') = ANY(mi.tags_search) THEN 4 -``` - -#### Change 3: Update SearchMediaItemsFuzzy (Lines 392-401) - -**Current:** -```sql -EXISTS ( - SELECT 1 FROM unnest(mi.tags) AS tag - WHERE word_similarity(sqlc.narg('search_query'), tag) > 0.3 - LIMIT 1 -) OR -EXISTS ( - SELECT 1 FROM unnest(mi.contributors) AS contributor - WHERE word_similarity(sqlc.narg('search_query'), contributor) > 0.3 - LIMIT 1 -) -``` - -**Replace with:** -```sql -EXISTS ( - SELECT 1 FROM unnest(mi.tags_search) AS tag - WHERE word_similarity(sqlc.narg('search_query'), tag) > 0.3 - LIMIT 1 -) OR -EXISTS ( - SELECT 1 FROM unnest(mi.contributors_search) AS contributor - WHERE word_similarity(sqlc.narg('search_query'), contributor) > 0.3 - LIMIT 1 -) -``` - -#### Change 4: Update SearchMediaItemsFuzzy ranking (Lines 408-417) - -**Current:** -```sql -COALESCE( - (SELECT MAX(word_similarity(sqlc.narg('search_query'), tag)) - FROM unnest(mi.tags) AS tag), - 0 -), -COALESCE( - (SELECT MAX(word_similarity(sqlc.narg('search_query'), contributor)) - FROM unnest(mi.contributors) AS contributor), - 0 -) -``` - -**Replace with:** -```sql -COALESCE( - (SELECT MAX(word_similarity(sqlc.narg('search_query'), tag)) - FROM unnest(mi.tags_search) AS tag), - 0 -), -COALESCE( - (SELECT MAX(word_similarity(sqlc.narg('search_query'), contributor)) - FROM unnest(mi.contributors_search) AS contributor), - 0 -) -``` - ---- - -## Phase 9: Frontend Documentation - -### File: `docs/FRONTEND_INTEGRATION.md` (NEW FILE) - -**Action:** Create frontend integration documentation - -**Create new file with content:** - -```markdown -# Frontend Integration Notes - -## Tag & Contributor Normalization - -The backend implements dual-field normalization for searchability: - -### Architecture - -| Field Type | Purpose | Behavior | Example | -|-----------|---------|-----------|----------| -| **Display Field** (`tags`, `contributors`) | Show to users | Preserves exact variant, punctuation, proper casing | `"ACME CORP."` | -| **Search Field** (`tags_search`, `contributors_search`) | Search against | Lowercase, no punctuation, deduplicated | `["acme corp"]` | - -### Normalization Rules - -#### Tags -1. Trim whitespace from each tag -2. Titlecase each tag (preserves hyphenation: "non-fiction" → "Non-Fiction") -3. Case-insensitive deduplication -4. Remove punctuation for search field only -5. Store both display and search versions - -#### Contributors -1. Trim whitespace from each contributor -2. Preserve original casing (including CAPSLOCK companies) -3. Preserve original punctuation for display -4. Remove punctuation for search comparison only -5. Case-insensitive deduplication -6. Store both display and search versions - -### API Request/Response - -**Request:** -```json -{ - "tags": ["science-fiction", "ACME CORP.", " O'Reilly Media"], - "contributors": [" Acme Corp ", "acme corp"] -} -``` - -**Response (after normalization):** -```json -{ - "tags": ["Science-Fiction", "O'Reilly Media"], - "tags_search": ["science fiction", "oreilly media"], - "contributors": ["Acme Corp", "acme corp"], - "contributors_search": ["acme corp"] -} -``` - -### Frontend Implementation Guidelines - -#### Display -- Use `tags` and `contributors` fields -- These preserve exact user input (casing, punctuation) -- No transformation needed - -#### Search -- Use search inputs against `tags_search` and `contributors_search` -- Normalize user search input: - - Convert to lowercase - - Remove punctuation (optional but recommended) - - Search using `= ANY()` operator - -#### User Typing "Science-Fiction" -```typescript -// User types exact value -const searchValue = "Science-Fiction"; - -// Backend normalizes to display and search versions -// Display: "Science-Fiction" -// Search: "science-fiction" -``` - -#### Search Query Behavior -```typescript -// User searches: "ACME CORP." -// Backend normalizes search to: "acme corp" -// This matches contributors_search = ["acme corp"] -// Which finds display contributors = ["ACME CORP.", "Acme Corp", "acme corp"] -``` - -### Checkbox Filter Integration - -When building frontend checkbox filters for contributors/tags: - -#### Get Unique Values for Dropdown -```typescript -// Fetch distinct normalized values for filters -GET /api/contributors?distinct=true -Response: ["acme corp", "oreilly media", "penguin"] - -// Render as checkboxes (using display names from another endpoint or mapping) -``` - -#### Filter Query -```typescript -// User selects checkbox -const filterValue = "acme corp"; - -// API request (filter by search field) -{ - "contributors_search": ["acme corp"] -} - -// Backend matches contributors_search array using = ANY() -``` - -### Important Notes - -1. **Display ≠ Search**: Always send search queries to search fields, not display fields -2. **Backend Normalization**: Backend normalizes input on CREATE/UPDATE, so always use search fields for filtering -3. **Case Sensitivity**: Search is case-insensitive, display is case-preserved -4. **Punctuation**: Display preserves it, search ignores it -5. **Deduplication**: Search fields are deduplicated, display fields are not - -### Common Mistakes to Avoid - -❌ **Searching display field directly** -```typescript -// WRONG - Will miss different casing/punctuation -WHERE 'ACME CORP.' = ANY(contributors) -``` - -✅ **Search search field** -```typescript -// CORRECT - Case-insensitive, punctuation-free -WHERE 'acme corp' = ANY(contributors_search) -``` - -❌ **Don't normalize user search input** -```typescript -// WRONG - If user types "ACME CORP" explicitly to find exact match -const search = "acme corp"; // Changes user's intent -``` - -✅ **Use exact user input for search** -```typescript -// CORRECT - Backend handles normalization -const search = "ACME CORP"; // Backend will match "acme corp" in search field -``` - -### Schema Reference - -**Display Fields:** -- `tags TEXT[]` - Titlecase, original punctuation -- `contributors TEXT[]` - Original casing, original punctuation - -**Search Fields:** -- `tags_search TEXT[]` - Lowercase, no punctuation, deduplicated -- `contributors_search TEXT[]` - Lowercase, no punctuation, deduplicated - -**GIN Indexes:** -- `idx_media_items_tags_search` - Fast search on tags_search -- `idx_media_items_contributors_search` - Fast search on contributors_search -- `idx_media_items_tags_gin` - Display field (if needed) -- `idx_media_items_contributors_gin` - Display field (if needed) - -### Example Flow - -1. **User creates media item:** - - Input: `tags: ["science-fiction", "ACME CORP."]` - - Backend stores: - - `tags`: `["Science-Fiction"]` (titlecased) - - `tags_search`: `["science fiction"]` (lowercase, no punctuation) - - `contributors`: `["ACME CORP."]` (preserved) - - `contributors_search`: `["acme corp"]` (normalized) - -2. **User searches "ACME CORP":** - - Frontend sends: `q: "ACME CORP"` - - Backend searches `tags_search` and `contributors_search` - - Finds: `contributors_search = ["acme corp"]` → MATCH ✅ - - Returns: Media item with `contributors = ["ACME CORP."]` - -3. **User searches "acme corp":** - - Frontend sends: `q: "acme corp"` - - Backend searches `tags_search` and `contributors_search` - - Finds: `contributors_search = ["acme corp"]` → MATCH ✅ - - Returns: Media item with `contributors = ["ACME CORP."]` - -4. **User searches "science-fiction":** - - Frontend sends: `q: "science-fiction"` - - Backend searches `tags_search` - - Finds: `tags_search = ["science fiction"]` → NO MATCH (hyphen vs space) - - Does NOT return (but fuzzy search might catch it) - -5. **User searches "science fiction":** - - Frontend sends: `q: "science fiction"` - - Backend searches `tags_search` - - Finds: `tags_search = ["science fiction"]` → MATCH ✅ - - Returns: Media item with `tags = ["Science-Fiction"]` -``` - ---- - -## Phase 10: Verification - -### Step 1: Verify Dependencies -```bash -go mod tidy -``` - -**Expected:** No errors - -### Step 2: Verify Compilation -```bash -go build ./cmd/server -``` - -**Expected:** Compiles without errors - -### Step 3: Verify Schema Changes -```bash -grep -n "tags_search\|contributors_search" database/schema/schema.sql -``` - -**Expected:** Shows 4 ALTER TABLE statements and 2 CREATE INDEX statements - -### Step 4: Run Unit Tests -```bash -go test ./internal/utils/... -v -``` - -**Expected:** All tests pass (100+ test cases) - -### Step 5: Verify sqlc Generation -```bash -cd internal/database && sqlc generate -``` - -**Expected:** No errors, models.go includes new fields - -### Step 6: Verify Handlers -```bash -go build ./cmd/server -``` - -**Expected:** Compiles successfully, no LSP errors - -### Step 7: Test Search Functionality -```bash -# Start server -DATABASE_PASSWORD=$(grep DBPASS .env | cut -d= -f2) go run ./cmd/server - -# Create test media item with tags/contributors -# Search for those tags/contributors with different casing/punctuation -``` - -**Expected:** Search works regardless of case or punctuation - ---- - -## Summary of Changes - -### Files Modified (9 files) - -1. `go.mod` - Add dependency -2. `database/schema/schema.sql` - Add 4 columns + 2 indexes -3. `internal/utils/tags.go` - Complete rewrite with 4 new functions -4. `internal/utils/tags_test.go` - NEW FILE (100+ tests) -5. `internal/database/queries/queries.sql` - Update 4 search queries -6. `internal/handlers/media.go` - Update 4 handler functions -7. `internal/services/ebook_scanner.go` - Update 2 scanner functions -8. `docs/FRONTEND_INTEGRATION.md` - NEW FILE (frontend docs) - -### Lines Changed -- **~50 lines** in handlers (normalization calls) -- **~30 lines** in scanner (normalization calls) -- **~20 lines** in queries.sql (search field updates) -- **~300 lines** in utils/tags.go (normalization functions) -- **~500 lines** in tags_test.go (test cases) - -### No Breaking Changes To - -- Column names (same tags/contributors for display) -- JSON field names (same tags/contributors for display) -- Other database columns -- Other API endpoints -- Function signatures (only added parameters) - -### New Columns Added - -- `tags_search TEXT[]` -- `contributors_search TEXT[]` -- 2 GIN indexes for search performance - ---- - -## Implementation Checklist - -- [ ] Dependencies added (go.mod) -- [ ] Schema updated (schema.sql) -- [ ] Normalization functions created (tags.go) -- [ ] Tests created (tags_test.go) -- [ ] sqlc regenerated (models.go, queries.sql.go) -- [ ] CreateMediaItem updated -- [ ] UpdateMediaItem updated -- [ ] HandleBulkUpdate updated -- [ ] Scanner updated (extractEPUBMetadata) -- [ ] Scanner updated (processEbookFile) -- [ ] Search queries updated (SearchMediaItems) -- [ ] Search queries updated (SearchMediaItemsFuzzy) -- [ ] Frontend documentation created (FRONTEND_INTEGRATION.md) -- [ ] Unit tests pass -- [ ] Compilation succeeds -- [ ] Schema changes verified - ---- - -## Notes - -- **No production data** - App never deployed, no migration needed -- **All phases independent** - Can stop after any phase if needed -- **Line numbers are approximate** - Verify before editing -- **Tests are comprehensive** - 100+ test cases cover all edge cases -- **Search is case-insensitive** - Works regardless of user input casing -- **Display preserves original** - Shows exact user input with proper formatting -- **Punctuation handling** - Display keeps it, search removes it