# Unified Search and Filter Implementation Plan ## Executive Summary **Goal:** Consolidate `/api/media-items/filtered` and `/api/media-items/search` endpoints into a single unified `/api/media-items/search` endpoint that supports: - All-fuzzy filters (except years/booleans) - Google-style "exact match in quotes" for search queries - Field-specific fuzzy search for autocomplete dropdowns - Combined search + filters functionality - Backward compatibility with saved filters **Approach:** Surgical, incremental changes that reuse existing code, following PROJECT_GUIDELINES.md strictly. --- ## Current State Analysis ### Existing Endpoints #### 1. `/api/media-items/search` (internal/router/search.go:11) - **Purpose:** Global fuzzy search across all libraries - **Handler:** `SearchMediaItems` (internal/handlers/media.go:1419-1493) - **Logic:** - Uses `SearchMediaItems` query (ILIKE pattern matching) - Falls back to `SearchMediaItemsFuzzy` query (pg_trgm `word_similarity()`) - Threshold: 0.3 similarity - **Parameters:** `q`, `library_id`, `limit`, `offset` - **Used by:** Search box (incorrectly - currently calls `/filtered`) #### 2. `/api/media-items/filtered` (internal/router/media.go:17) - **Purpose:** Exact match filtering - **Handler:** `ListMediaItemsFiltered` (internal/handlers/media.go:705-766) - **Logic:** - `ListMediaItemsFiltered` query (queries.sql:227-268) - Author/series: `ILIKE` (case-insensitive, NO wildcards) - Genre/language: `=` (exact match) - Years: `>=`, `<=` (exact range) - Boolean: exact match - **Parameters:** `author_filter`, `series_filter`, `genre_filter`, `language_filter`, `year_min`, `year_max`, `has_cover`, `sort`, `limit`, `offset` - **Used by:** All filter fields, search box (incorrectly) ### Current Frontend Implementation **Search box** (templates/bookshelf.templ:73-83): ```html hx-trigger="keyup changed delay:300ms" hx-include="#filter-form"> ``` **Filter fields** (templates/bookshelf.templ:90-117): ```html ``` ### Database Schema **Indexes exist** (database/schema/schema.sql): - B-tree indexes: author, genre, language, series, copyright_year (lines 365-373) - GIN indexes: tags_search, contributors_search (lines 145-146) **Missing:** GIN indexes for pg_trgm fuzzy search on text fields ### Saved Filters **Implementation:** Frontend-only (web/src/bookshelf.ts:44-173) - Stores filter config as JSON in database - Loads filter config into form via `loadFilter()` method - Makes API call to `/api/saved-filters/:id` - Populates form fields with exact values **Impact:** Should work seamlessly with new endpoint (only field names matter) --- ## Implementation Phases ### Phase 1: Database Schema - Add GIN Indexes **File:** `database/schema/schema.sql` **Add after line 146:** ```sql -- Add GIN indexes for pg_trgm fuzzy search performance CREATE INDEX IF NOT EXISTS idx_media_items_author_trgm ON media_items USING GIN (author gin_trgm_ops); CREATE INDEX IF NOT EXISTS idx_media_items_title_trgm ON media_items USING GIN (title gin_trgm_ops); CREATE INDEX IF NOT EXISTS idx_media_items_series_trgm ON media_items USING GIN (series gin_trgm_ops); CREATE INDEX IF NOT EXISTS idx_media_items_genre_trgm ON media_items USING GIN (genre gin_trgm_ops); CREATE INDEX IF NOT EXISTS idx_media_items_language_trgm ON media_items USING GIN (language gin_trgm_ops); ``` **Verification:** ```bash podman compose down -v podman compose up -d podman exec bookhoard_db psql -U postgres -d bookhoard -c "\d+ media_items" ``` --- ### Phase 2: SQL Queries - Add Unified Search Query **File:** `internal/database/queries/queries.sql` **Add after `SearchMediaItemsFuzzy` (line 460):** ```sql -- name: SearchMediaItemsUnified :many SELECT mi.*, l.name as library_name, lt.name as library_type_name FROM media_items mi JOIN libraries l ON mi.library_id = l.id JOIN library_types lt ON l.library_type_id = lt.id LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id') WHERE COALESCE(lv.is_visible, true) = true AND mi.library_id = sqlc.narg('library_id') -- Fuzzy author filter AND (sqlc.narg('author_filter') = '' OR word_similarity(sqlc.narg('author_filter'), COALESCE(mi.author, '')) > 0.3) -- Fuzzy series filter AND (sqlc.narg('series_filter') = '' OR word_similarity(sqlc.narg('series_filter'), COALESCE(mi.series, '')) > 0.3) -- Fuzzy genre filter AND (sqlc.narg('genre_filter') = '' OR word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')) > 0.3) -- Fuzzy language filter AND (sqlc.narg('language_filter') = '' OR word_similarity(sqlc.narg('language_filter'), COALESCE(mi.language, '')) > 0.3) -- Year range (exact) AND (sqlc.narg('year_min') = 0 OR mi.copyright_year >= sqlc.narg('year_min')) AND (sqlc.narg('year_max') = 0 OR mi.copyright_year <= sqlc.narg('year_max')) -- Boolean (exact) AND (sqlc.narg('has_cover') = false OR mi.cover_image_path IS NOT NULL) -- Search query (fuzzy or exact based on quotes) AND ( sqlc.narg('search_query') = '' OR -- Fuzzy search (default) sqlc.narg('is_exact_search') = false AND ( word_similarity(sqlc.narg('search_query'), mi.title) > 0.3 OR word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, '')) > 0.3 OR word_similarity(sqlc.narg('search_query'), COALESCE(mi.series, '')) > 0.3 OR EXISTS ( SELECT 1 FROM unnest(mi.tags_search) AS tag WHERE word_similarity(sqlc.narg('search_query'), tag) > 0.3 LIMIT 1 ) OR EXISTS ( SELECT 1 FROM unnest(mi.contributors_search) AS contributor WHERE word_similarity(sqlc.narg('search_query'), contributor) > 0.3 LIMIT 1 ) ) OR -- Exact search (with quotes) sqlc.narg('is_exact_search') = true AND ( mi.title ILIKE sqlc.narg('search_pattern') OR mi.author ILIKE sqlc.narg('search_pattern') OR mi.series ILIKE sqlc.narg('search_pattern') OR sqlc.narg('search_pattern') = ANY(mi.tags_search) OR sqlc.narg('search_pattern') = ANY(mi.contributors_search) ) ) ORDER BY CASE WHEN sqlc.narg('search_query') != '' THEN GREATEST( CASE WHEN sqlc.narg('is_exact_search') = false THEN word_similarity(sqlc.narg('search_query'), mi.title) ELSE 0 END, CASE WHEN sqlc.narg('is_exact_search') = false THEN word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, '')) ELSE 0 END, word_similarity(sqlc.narg('author_filter'), COALESCE(mi.author, '')), word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')) ) ELSE 0 END DESC, mi.title ASC LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset'); ``` **Add field value search query (for autocomplete dropdowns):** ```sql -- name: SearchFieldValues :many SELECT DISTINCT CASE sqlc.narg('field_type') WHEN 'author' THEN mi.author WHEN 'genre' THEN mi.genre WHEN 'series' THEN mi.series WHEN 'language' THEN mi.language END as value, COUNT(*) as count, CASE sqlc.narg('field_type') WHEN 'author' THEN word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, '')) WHEN 'genre' THEN word_similarity(sqlc.narg('search_query'), COALESCE(mi.genre, '')) WHEN 'series' THEN word_similarity(sqlc.narg('search_query'), COALESCE(mi.series, '')) WHEN 'language' THEN word_similarity(sqlc.narg('search_query'), COALESCE(mi.language, '')) END as score FROM media_items mi WHERE mi.library_id = sqlc.narg('library_id') AND ( (sqlc.narg('field_type') = 'author' AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, '')) > 0.3) OR (sqlc.narg('field_type') = 'genre' AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.genre, '')) > 0.3) OR (sqlc.narg('field_type') = 'series' AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.series, '')) > 0.3) OR (sqlc.narg('field_type') = 'language' AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.language, '')) > 0.3) ) GROUP BY value, score HAVING value IS NOT NULL AND value != '' ORDER BY score DESC, count DESC LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset'); ``` **Regenerate Go code:** ```bash go generate ./internal/database ``` **Verify:** Check `internal/database/queries.sql.go` for new functions --- ### Phase 3: Create Search Service **File:** `internal/services/search.go` (NEW) ```go package services import ( "bookhoard/internal/database" "context" "strconv" "strings" "github.com/google/uuid" "github.com/jackc/pgx/v5/pgtype" ) // SearchService handles all search and filter operations type SearchService struct { db *database.Queries } // NewSearchService creates a new search service instance func NewSearchService(db *database.Queries) *SearchService { return &SearchService{db: db} } // SearchParams contains parameters for unified search type SearchParams struct { UserID pgtype.UUID LibraryID pgtype.UUID AuthorFilter string SeriesFilter string GenreFilter string LanguageFilter string YearMin int YearMax int HasCover bool SearchQuery string Limit int Offset int } // parseSearchQuery detects quoted strings for exact match search // Returns (isExact, processedQuery) // Examples: // "asimov" → (false, "asimov") // "\"Asimov, Isaac\"" → (true, "Asimov, Isaac") func (s *SearchService) parseSearchQuery(query string) (bool, string) { query = strings.TrimSpace(query) if strings.HasPrefix(query, "\"") && strings.HasSuffix(query, "\"") && len(query) >= 2 { return true, strings.Trim(query, "\"") } return false, query } // SearchMediaItemsUnified handles combined search + filters // Supports: // - Fuzzy text filters (author, series, genre, language) // - Exact filters (year range, has_cover boolean) // - Fuzzy search query (or exact match with quotes) // - Combined search + filters func (s *SearchService) SearchMediaItemsUnified(ctx context.Context, params SearchParams) ([]database.SearchMediaItemsUnifiedRow, error) { // Parse search query for exact match detection isExact, searchQuery := s.parseSearchQuery(params.SearchQuery) searchPattern := "" if isExact { searchPattern = "%" + searchQuery + "%" } // Build database parameters dbParams := database.SearchMediaItemsUnifiedParams{ UserID: params.UserID, LibraryID: params.LibraryID, AuthorFilter: pgtype.Text{String: params.AuthorFilter, Valid: true}, SeriesFilter: pgtype.Text{String: params.SeriesFilter, Valid: true}, GenreFilter: pgtype.Text{String: params.GenreFilter, Valid: true}, LanguageFilter: pgtype.Text{String: params.LanguageFilter, Valid: true}, YearMin: pgtype.Int4{Int32: int32(params.YearMin), Valid: true}, YearMax: pgtype.Int4{Int32: int32(params.YearMax), Valid: true}, HasCover: pgtype.Bool{Bool: params.HasCover, Valid: true}, SearchQuery: pgtype.Text{String: searchQuery, Valid: true}, IsExactSearch: pgtype.Bool{Bool: isExact, Valid: true}, SearchPattern: pgtype.Text{String: searchPattern, Valid: isExact}, Limit: pgtype.Int4{Int32: int32(params.Limit), Valid: true}, Offset: pgtype.Int4{Int32: int32(params.Offset), Valid: true}, } // Execute unified search query results, err := s.db.SearchMediaItemsUnified(ctx, dbParams) if err != nil { return nil, err } return results, nil } // FieldSearchParams contains parameters for field-specific search (autocomplete) type FieldSearchParams struct { UserID pgtype.UUID LibraryID pgtype.UUID FieldType string // "author", "genre", "series", "language" SearchQuery string Limit int Offset int } // FieldValue represents a single field value with metadata type FieldValue struct { Value string Count int64 Score float64 } // SearchFieldValues handles field-specific search for autocomplete dropdowns // Returns distinct values with counts and similarity scores func (s *SearchService) SearchFieldValues(ctx context.Context, params FieldSearchParams) ([]FieldValue, error) { // Build database parameters dbParams := database.SearchFieldValuesParams{ UserID: params.UserID, LibraryID: params.LibraryID, FieldType: pgtype.Text{String: params.FieldType, Valid: true}, SearchQuery: pgtype.Text{String: params.SearchQuery, Valid: true}, Limit: pgtype.Int4{Int32: params.Limit), Valid: true}, Offset: pgtype.Int4{Int32: params.Offset), Valid: true}, } // Execute field values query results, err := s.db.SearchFieldValues(ctx, dbParams) if err != nil { return nil, err } // Convert to service-level response type fieldValues := make([]FieldValue, len(results)) for i, r := range results { fieldValues[i] = FieldValue{ Value: r.Value, Count: r.Count, Score: r.Score, } } return fieldValues, nil } ``` **Verification:** ```bash go build ./internal/handlers ``` **Note:** Router initialization will also need updating (see Phase 7) #### 4.1 Fix Search Box Endpoint **File:** `templates/bookshelf.templ` **Change line 79:** ```html ``` **Also fix pagination buttons (lines 298, 311):** ```html hx-get="/api/media-items/filtered?library_id={ currentLibraryID }&limit={ limit }&offset={ offset - limit }" hx-get="/api/media-items/search?library_id={ currentLibraryID }&limit={ limit }&offset={ offset - limit }" ``` **Change filter field names (lines 92, 109, 126, etc.):** ```html ``` #### 4.2 Add Autocomplete Dropdowns **File:** `templates/bookshelf.templ` **Replace author filter (lines 85-101):** ```html
``` **Similar changes for genre, series, language filters** #### 4.3 Add Frontend Functions **File:** `web/src/bookshelf.ts` **Add at end of file:** ```typescript // Fetch field values for autocomplete dropdowns async function fetchFieldValues( field: string, search: string, datalistId: string ): Promise { const token = localStorage.getItem("token"); if (!token) { console.error("Not authenticated"); return; } if (search.length < 2) return; // Wait for at least 2 characters const currentLibraryId = ( document.getElementById("library-select") as HTMLSelectElement )?.value; if (!currentLibraryId) { console.error("No library selected"); return; } try { const response = await fetch( `/api/media-items/search?${field}=${encodeURIComponent( search )}&library_id=${currentLibraryId}&limit=50`, { headers: { Authorization: `Bearer ${token}` }, } ); if (!response.ok) { console.error("Failed to fetch field values"); return; } const data = await response.json(); // Update datalist const datalist = document.getElementById(datalistId); if (!datalist) { console.error(`Datalist ${datalistId} not found`); return; } // Clear existing options datalist.innerHTML = ""; // Add new options data.results.forEach((item: { value: string; count: number }) => { const option = document.createElement("option"); option.value = item.value; option.textContent = `${item.value} (${item.count})`; datalist.appendChild(option); }); } catch (error) { console.error("Error fetching field values:", error); } } // Fetch author values for autocomplete async function fetchAuthorValues(input: HTMLInputElement): Promise { const search = input.value; await fetchFieldValues("authors", search, "author-datalist"); } // Fetch genre values for autocomplete async function fetchGenreValues(input: HTMLInputElement): Promise { const search = input.value; await fetchFieldValues("genres", search, "genre-datalist"); } // Fetch series values for autocomplete async function fetchSeriesValues(input: HTMLInputElement): Promise { const search = input.value; await fetchFieldValues("series", search, "series-datalist"); } // Fetch language values for autocomplete async function fetchLanguageValues(input: HTMLInputElement): Promise { const search = input.value; await fetchFieldValues("languages", search, "language-datalist"); } // Register functions globally (window as any).fetchAuthorValues = fetchAuthorValues; (window as any).fetchGenreValues = fetchGenreValues; (window as any).fetchSeriesValues = fetchSeriesValues; (window as any).fetchLanguageValues = fetchLanguageValues; ``` **Verify TypeScript compilation:** ```bash cd web && npm run build ``` --- ### Phase 5: Tests **File:** `cmd/server/tests/search_unified_test.go` (NEW) ```go package main import ( "bookhoard/internal/handlers" "encoding/json" "net/http" "net/http/httptest" "testing" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" ) func TestUnifiedSearch(t *testing.T) { setup := setupDeviceTest(t) defer setup.Server.Close() libraryID := setup.CreateLibrary(t, "Test Search Library", "ebooks") _ = setup.CreateDevice(t, "Test Search Device", "koreader", "search-test-123") t.Run("Fuzzy author filter", func(t *testing.T) { req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&author_filter=asimov", nil) req.Header.Set("Authorization", "Bearer "+setup.UserToken) rec := httptest.NewRecorder() setup.Server.Config.Handler.ServeHTTP(rec, req) assert.Equal(t, http.StatusOK, rec.Code, "Should fuzzy match author") }) t.Run("Fuzzy genre filter", func(t *testing.T) { req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&genre_filter=scifi", nil) req.Header.Set("Authorization", "Bearer "+setup.UserToken) rec := httptest.NewRecorder() setup.Server.Config.Handler.ServeHTTP(rec, req) assert.Equal(t, http.StatusOK, rec.Code, "Should fuzzy match genre") }) t.Run("Exact match with quotes", func(t *testing.T) { req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&q=%22Foundation%20and%20Empire%22", nil) req.Header.Set("Authorization", "Bearer "+setup.UserToken) rec := httptest.NewRecorder() setup.Server.Config.Handler.ServeHTTP(rec, req) assert.Equal(t, http.StatusOK, rec.Code, "Should exact match quoted query") }) t.Run("Combined search + filters", func(t *testing.T) { req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&q=foundation&author_filter=asimov", nil) req.Header.Set("Authorization", "Bearer "+setup.UserToken) rec := httptest.NewRecorder() setup.Server.Config.Handler.ServeHTTP(rec, req) assert.Equal(t, http.StatusOK, rec.Code, "Should combine search and filters") }) t.Run("Field-specific search for dropdown - authors", func(t *testing.T) { req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&authors=asimov", nil) req.Header.Set("Authorization", "Bearer "+setup.UserToken) rec := httptest.NewRecorder() setup.Server.Config.Handler.ServeHTTP(rec, req) assert.Equal(t, http.StatusOK, rec.Code, "Should return author values") var response struct { Results []struct { Value string `json:"value"` Count int64 `json:"count"` Score float64 `json:"score"` } `json:"results"` Total int `json:"total"` } err := json.Unmarshal(rec.Body.Bytes(), &response) require.NoError(t, err, "Should unmarshal field values response") assert.Greater(t, len(response.Results), 0, "Should have results") }) t.Run("Year range filter (exact)", func(t *testing.T) { req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&year_min=2000&year_max=2020", nil) req.Header.Set("Authorization", "Bearer "+setup.UserToken) rec := httptest.NewRecorder() setup.Server.Config.Handler.ServeHTTP(rec, req) assert.Equal(t, http.StatusOK, rec.Code, "Should filter by year range") }) t.Run("Boolean filter (exact)", func(t *testing.T) { req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&has_cover=true", nil) req.Header.Set("Authorization", "Bearer "+setup.UserToken) rec := httptest.NewRecorder() setup.Server.Config.Handler.ServeHTTP(rec, req) assert.Equal(t, http.StatusOK, rec.Code, "Should filter by has_cover") }) t.Run("Missing library_id", func(t *testing.T) { req := httptest.NewRequest("GET", "/api/media-items/search?q=test", nil) req.Header.Set("Authorization", "Bearer "+setup.UserToken) rec := httptest.NewRecorder() setup.Server.Config.Handler.ServeHTTP(rec, req) assert.Equal(t, http.StatusBadRequest, rec.Code, "Should require library_id") }) } ``` **Verify tests:** ```bash go test ./cmd/server/tests -v -run TestUnifiedSearch ``` --- ### Phase 6: Documentation Updates #### 6.1 Update API Documentation **File:** `docs/developer/api/media-items/search_media_items.md` **Replace entire content:** ```markdown # Search Media Items (Unified) Search and filter media items with fuzzy matching support. **Note:** All text filters use fuzzy matching via PostgreSQL pg_trgm (threshold: 0.3 similarity). This handles typos and partial matches automatically. Use quotes for exact match. **Endpoint**: `GET /api/media-items/search` **Auth**: Required ## Query Parameters ### Search Parameters | Parameter | Type | Required | Description | | ---------- | ------- | -------- | ---------------------------------------------------- | | q | string | No | Search query (fuzzy by default, exact in quotes) | | library_id | string | Yes | Filter to specific library (UUID) | | limit | integer | No | Number of results (default 50, max 200) | | offset | integer | No | Number to skip for pagination | ### Filter Parameters (All Fuzzy Except Years/Booleans) | Parameter | Type | Description | | -------------- | ------- | --------------------------------------------------- | | author_filter | string | Fuzzy match author field | | series_filter | string | Fuzzy match series field | | genre_filter | string | Fuzzy match genre field | | language_filter| string | Fuzzy match language field | | year_min | integer | Minimum copyright year (exact range) | | year_max | integer | Maximum copyright year (exact range) | | has_cover | boolean | Filter by cover image presence (exact boolean) | ### Autocomplete Parameters (Field-Specific Search) | Parameter | Type | Description | | ---------- | ------ | ---------------------------------------------- | | authors | string | Search author values for autocomplete dropdown | | genres | string | Search genre values for autocomplete dropdown | | series | string | Search series values for autocomplete dropdown | | languages | string | Search language values for autocomplete | ## Request Headers | Header | Type | Required | Description | | ------------- | ------ | -------- | ------------ | | Authorization | string | Yes | Bearer token | ## Search Behavior ### Fuzzy Search (Default) Handles typos and partial matches automatically: - `"asimov"` → matches "Asimov, Isaac", "Asimov, Foundation" - `"scifi"` → matches "Sci-Fi", "Science Fiction" - `"azimov"` → matches "Asimov, Isaac" (typo tolerance) ### Exact Search (With Quotes) Use double quotes for exact phrase matching: - `"\"Foundation and Empire\""` → only "Foundation and Empire" - `"\"Asimov, Isaac\""` → only "Asimov, Isaac" ### Filter Behavior **Text filters (fuzzy):** - `author_filter=asimov` → fuzzy matches author field - `genre_filter=scifi` → fuzzy matches genre field **Exact filters:** - `year_min=2000&year_max=2010` → exact year range - `has_cover=true` → exact boolean match ## Example Requests ### 1. Global Fuzzy Search Search all fields for "foundation": ```http GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&q=foundation Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... ``` ### 2. Fuzzy Author Filter Find books by "asimov" (matches "Asimov, Isaac"): ```http GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&author_filter=asimov Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... ``` ### 3. Combined Search + Filters Search "foundation" within books by "asimov": ```http GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&q=foundation&author_filter=asimov Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... ``` ### 4. Exact Match with Quotes Exact phrase search: ```http GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&q="Foundation%20and%20Empire" Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... ``` ### 5. Multiple Fuzzy Filters Fiction books from 2000-2010: ```http GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&genre_filter=fiction&year_min=2000&year_max=2010 Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... ``` ### 6. Field-Specific Search (Autocomplete) Get author values for dropdown: ```http GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&authors=asimov&limit=50 Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... ``` **Response:** ```json { "results": [ {"value": "Asimov, Isaac", "count": 47, "score": 0.8}, {"value": "Asimov, Isaac & Robert Silverberg", "count": 2, "score": 0.75} ], "total": 2 } ``` ## Response (200 OK) **Media items search:** ```json [ { "id": "uuid", "title": "Foundation", "author": "Asimov, Isaac", "library_id": "...", "library_name": "E-Books" } ] ``` **Field values search (autocomplete):** ```json { "results": [ {"value": "Asimov, Isaac", "count": 47, "score": 0.8} ], "total": 1 } ``` ## Error Responses | Code | Description | | ---- | ---------------------------- | | 400 | Invalid library_id | | 400 | Missing library_id | | 401 | Invalid or expired token | | 404 | No results found | ``` #### 6.2 Update Bruno Collection **File:** `bruno/media-items/Search All Libraries.yml` **Update params section:** ```yaml params: - name: q value: "foundation" type: query disabled: false - name: library_id value: "{{library_id}}" type: query disabled: false - name: author_filter value: "" type: query disabled: true - name: genre_filter value: "" type: query disabled: true - name: year_min value: "" type: query disabled: true - name: year_max value: "" type: query disabled: true ``` **Add new Bruno files:** `bruno/media-items/Fuzzy Author Filter.yml` `bruno/media-items/Fuzzy Genre Filter.yml` `bruno/media-items/Combined Search and Filters.yml` `bruno/media-items/Exact Match With Quotes.yml` `bruno/media-items/Field Values Search - Authors.yml` `bruno/media-items/scenarios/Unified Search Scenarios.yml` #### 6.3 Delete Deprecated Filtered Endpoint Documentation **Files to delete:** - `docs/developer/api/media-items/filtered_media_items.md` (if exists) - Remove `/api/media-items/filtered` from `docs/developer/api/api-reference.md` - Remove `/api/media-items/filtered` from `docs/developer/api-reference.md` --- ### Phase 7: Update Handler Initialization **Files to update:** - `cmd/server/main.go` (line ~122-126) - `cmd/server/tests/test_helpers_test.go` (line ~468-472) **In both files, find the MediaHandler initialization:** ```go // BEFORE mediaHandler := handlers.NewMediaHandler(queries, libraryService, worker) // AFTER searchService := services.NewSearchService(queries) mediaHandler := handlers.NewMediaHandler(queries, libraryService, searchService, worker) ``` **Note:** This follows the same pattern as `conversionService` which is created in main.go (line 118) and passed to handlers. **Verify compilation:** ```bash go build ./cmd/server go test ./cmd/server/tests -run TestNonExistent # Compile test only ``` --- ### Phase 8: Cleanup #### 7.1 Delete Deprecated Filtered Endpoint **File:** `internal/router/media.go` **Delete line 17:** ```go protected.GET("/media-items/filtered", cfg.MediaHandler.ListMediaItemsFiltered) ``` **File:** `internal/handlers/media.go` **Delete handler `ListMediaItemsFiltered` (lines 705-766):** ```go // DELETE THIS FUNCTION func (mh *MediaHandler) ListMediaItemsFiltered(c *echo.Context) error { ... } ``` **File:** `internal/database/queries/queries.sql` **Delete query `ListMediaItemsFiltered` (lines 227-268):** ```sql -- DELETE THIS QUERY -- name: ListMediaItemsFiltered :many ... ``` **Regenerate Go code:** ```bash go generate ./internal/database ``` #### 7.2 Delete Deprecated Tests **File:** `cmd/server/tests/filtering_test.go` **Delete entire file** (tests now covered by `search_unified_test.go`) **OR** update tests to use `/api/media-items/search` endpoint if some test cases are still valuable #### 7.3 Delete Deprecated Bruno Files **Delete:** `bruno/media-items/scenarios/Filter Media Items.yml` (if exists) --- ### Phase 10: Verification #### 8.1 Compile Check ```bash go build ./... cd web && npm run build ``` #### 8.2 Run Tests ```bash go test ./cmd/server/tests -v -run TestUnifiedSearch go test ./cmd/server/tests -v # All tests should pass ``` #### 8.3 Manual Testing Checklist - [ ] Search box uses `/api/media-items/search` - [ ] Fuzzy author filter works (asimov → Asimov, Isaac) - [ ] Fuzzy genre filter works (scifi → Sci-Fi) - [ ] Exact match with quotes works ("Foundation and Empire") - [ ] Combined search + filters works - [ ] Autocomplete dropdowns populate correctly - [ ] Year range filter works (exact) - [ ] Boolean filter works (exact) - [ ] Saved filters load correctly - [ ] Pagination works - [ ] No errors in browser console - [ ] No errors in server logs #### 8.4 Documentation Verification ```bash # Start server podman compose up -d # Access docs at http://localhost:8080/docs # Verify search endpoint documentation renders correctly # Verify search finds new documentation ``` --- **Commit 3: Add search service** git add internal/services/search.go git commit -m "feat: add SearchService for unified search functionality - Create SearchService with SearchMediaItemsUnified method - Add SearchFieldValues method for autocomplete dropdowns - Add parseSearchQuery helper for quote detection - Move all business logic from handler to service layer - Follow established service pattern (FiltersService, CollectionService)" ```bash # Commit 1: Database schema (GIN indexes) git add database/schema/schema.sql git commit -m "feat: add GIN indexes for pg_trgm fuzzy search performance - Add gin_trgm_ops indexes on author, title, series, genre, language - Improves fuzzy search performance on large libraries - Required for unified search/filter endpoint" # Commit 2: SQL queries git add internal/database/queries/queries.sql git commit -m "feat: add unified search SQL query with fuzzy filters - Add SearchMediaItemsUnified query with all-fuzzy filters - Add SearchFieldValues query for autocomplete dropdowns - Support exact match with quotes detection - Combine search + filters in single query" # Commit 3: Add search service git add internal/services/search.go git commit -m "feat: add SearchService for unified search functionality - Create SearchService with SearchMediaItemsUnified method - Add SearchFieldValues method for autocomplete dropdowns - Add parseSearchQuery helper for quote detection - Move all business logic from handler to service layer - Follow established service pattern (FiltersService, CollectionService)" # Commit 4: Regenerate database code git add internal/database/queries.sql.go internal/database/models.go git commit -m "chore: regenerate database code from queries.sql" # Commit 5: Update handler to use search service git add internal/handlers/media.go git commit -m "refactor: update SearchMediaItems handler to use SearchService - Add searchService to MediaHandler struct - Update NewMediaHandler constructor - Refactor SearchMediaItems to delegate to service layer - Add handleUnifiedSearch method (thin wrapper) - Add handleFieldValuesSearch method (thin wrapper) - Handler now only extracts params and calls service" # Commit 6: Update handler initialization git add cmd/server/main.go cmd/server/tests/test_helpers_test.go git commit -m "refactor: add SearchService to handler initialization - Instantiate SearchService in main.go and test_helpers_test.go - Pass searchService to MediaHandler constructor - Follow existing pattern (like conversionService) - Update both production and test initialization" # Commit 7: Frontend templates git add templates/bookshelf.templ git commit -m "feat: fix search box to use /search endpoint with autocomplete - Change search box from /filtered to /search - Change pagination buttons to use /search - Add datalist elements for autocomplete - Add Alpine.js event handlers for dropdown population" # Commit 8: Frontend TypeScript git add web/src/bookshelf.ts git commit -m "feat: add autocomplete dropdown support for filter fields - Add fetchFieldValues function for API calls - Add fetchAuthorValues, fetchGenreValues, etc. - Register functions globally for template access - Populate datalist elements with fuzzy search results" # Commit 9: Tests git add cmd/server/tests/search_unified_test.go git commit -m "test: add comprehensive tests for unified search endpoint - Test fuzzy author/genre filters - Test exact match with quotes - Test combined search + filters - Test field-specific search for dropdowns - Test year range and boolean filters - Use setupTestServer helper following PROJECT_GUIDELINES.md" # Commit 10: Documentation git add docs/developer/api/media-items/search_media_items.md git add bruno/media-items/Search\ All\ Libraries.yml git add bruno/media-items/Fuzzy\ Author\ Filter.yml git add bruno/media-items/Fuzzy\ Genre\ Filter.yml git add bruno/media-items/Combined\ Search\ and\ Filters.yml git add bruno/media-items/Exact\ Match\ With\ Quotes.yml git add bruno/media-items/Field\ Values\ Search\ -\ Authors.yml git commit -m "docs: update search API documentation with fuzzy filters - Document all-fuzzy filters (except years/booleans) - Document exact match with quotes - Document combined search + filters - Document field-specific search for autocomplete - Add comprehensive examples - Update Bruno collection with new endpoints" # Commit 11: Delete deprecated code git add internal/router/media.go git add internal/handlers/media.go git add internal/database/queries/queries.sql git add internal/database/queries.sql.go git add internal/database/models.go git add cmd/server/tests/filtering_test.go git commit -m "refactor: remove deprecated /filtered endpoint - Delete /media-items/filtered route registration - Delete ListMediaItemsFiltered handler - Delete ListMediaItemsFiltered SQL query - Delete filtering_test.go (covered by search_unified_test.go) - Regenerate database code after query deletion" # Commit 12: Final verification git add . git commit -m "chore: final verification of unified search implementation - All tests pass - Documentation renders correctly - Bruno collection updated - No compilation errors - Manual testing complete" ``` --- ## Risk Mitigation ### Potential Issues 1. **Saved filters breaking:** Frontend-only, should work seamlessly 2. **Performance degradation:** GIN indexes should prevent this 3. **Breaking mobile apps:** `/filtered` endpoint will be deleted 4. **Test coverage gaps:** Comprehensive tests in Phase 5 ### Rollback Plan If issues arise: ```bash # Revert to previous commit git revert HEAD # Or restore specific files git show HEAD~1:internal/handlers/media.go > internal/handlers/media.go git show HEAD~1:templates/bookshelf.templ > templates/bookshelf.templ ``` --- ## Timeline Estimate - Phase 1 (Database): 30 minutes - Phase 2 (SQL Queries): 1 hour - Phase 3 (Search Service): 2 hours - Phase 4 (Handler Updates): 1 hour - Phase 5 (Router Updates): 15 minutes - Phase 6 (Frontend): 2 hours - Phase 7 (TypeScript): 1 hour - Phase 8 (Tests): 2 hours - Phase 9 (Documentation): 1 hour - Phase 10 (Cleanup): 30 minutes - Phase 11 (Verification): 1 hour **Total: ~12 hours** --- ## Success Criteria ✅ All business logic in `services/search.go` (service layer pattern) ✅ Handler is thin - only extracts params and calls service ✅ All text filters use fuzzy matching (pg_trgm) ✅ Exact match with quotes works ✅ Combined search + filters work ✅ Autocomplete dropdowns populate correctly ✅ Years/booleans remain exact match ✅ Saved filters load correctly ✅ All tests pass using `setupTestServer` helper ✅ Documentation updated ✅ No breaking changes to saved filters ✅ Deprecated `/filtered` endpoint removed ✅ Bruno collection updated