From a3fe47ac2119f08ee2a3fbfd680b13de70170817 Mon Sep 17 00:00:00 2001 From: John O'Keefe Date: Thu, 26 Mar 2026 14:38:31 -0400 Subject: [PATCH] Update and consolidate implementation documentation Clean up documentation by removing obsolete implementation notes and updating the Calibre OPF implementation guide. Changes: - Update CALIBRE_OPF_IMPLEMENTATION.md with namespace URL approach - Remove IMPLEMENTATION_TAGS_FILTER.md (superseded by unified search) - Remove UNIFIED_SEARCH_IMPLEMENTATION.md (implementation complete) The Calibre OPF documentation now reflects the corrected approach using full Dublin Core namespace URLs (http://purl.org/dc/elements/1.1/) instead of namespace prefixes, which were found to not work with Go's XML decoder. Documentation: #docs-cleanup --- CALIBRE_OPF_IMPLEMENTATION.md | 2 +- IMPLEMENTATION_TAGS_FILTER.md | 748 ------------ UNIFIED_SEARCH_IMPLEMENTATION.md | 1924 ------------------------------ 3 files changed, 1 insertion(+), 2673 deletions(-) delete mode 100644 IMPLEMENTATION_TAGS_FILTER.md delete mode 100644 UNIFIED_SEARCH_IMPLEMENTATION.md diff --git a/CALIBRE_OPF_IMPLEMENTATION.md b/CALIBRE_OPF_IMPLEMENTATION.md index 0475d6e..50b3680 100644 --- a/CALIBRE_OPF_IMPLEMENTATION.md +++ b/CALIBRE_OPF_IMPLEMENTATION.md @@ -748,7 +748,7 @@ func TestExtractCalibreSidecar(t *testing.T) { **ADD THIS INTEGRATION TEST** (optional, for comprehensive testing): ```go -package tests +package main import ( "context" diff --git a/IMPLEMENTATION_TAGS_FILTER.md b/IMPLEMENTATION_TAGS_FILTER.md deleted file mode 100644 index 4abfbdf..0000000 --- a/IMPLEMENTATION_TAGS_FILTER.md +++ /dev/null @@ -1,748 +0,0 @@ -# Implementation Plan: Replace Genre Filter with Tags Filter - -## Status: ๐Ÿ“‹ Planning Phase -**Created:** 2026-03-25 -**Type:** Full-Stack Task -**Priority:** Medium -**Estimated Time:** 2-3 hours -**Note:** Tags filter uses fuzzy matching (consistent with author/series filters) - ---- - -## ๐ŸŽฏ Overview - -Replace the `genre_filter` with `tags_filter` throughout the application. The `genre` field in the database is currently NULL for all Calibre imports because Calibre uses tags for genre classification. By filtering tags instead, we leverage existing data without requiring a schema migration. - -**Key Decision:** Keep `genre_filter` in the API for backward compatibility but remove it from the frontend UI. - ---- - -## ๐Ÿ“Š Current State Analysis - -### Database State -```sql --- Current data (from investigation): -SELECT COUNT(*) as total, COUNT(genre) as with_genre, COUNT(tags) as with_tags FROM media_items; --- Result: 17 total books, 0 with genre, 17 with tags -``` - -### Problem -- **Genre field:** NULL for all imported books (Calibre doesn't populate it) -- **Tags field:** Populated with Calibre tags (including genre-like tags) -- **Filter behavior:** genre_filter returns 0 results (no matches on NULL) -- **User impact:** Cannot filter books by genre/category - -### Root Cause -Calibre treats tags as genre classification. Example tags: -``` -["Science Fiction", "Adventure", "Dystopian", "Favorites"] -``` - -The current genre filter expects a single `genre` text field, which is always NULL. - ---- - -## ๐Ÿ”„ Proposed Solution - -### Option B: Filter Tags Instead (Selected) - -**Why this approach:** -- โœ… No database migration required (zero downtime) -- โœ… Works immediately with existing Calibre data -- โœ… Low risk (isolated changes) -- โœ… Calibre users already understand this model -- โœ… Faster to implement - -**Trade-offs:** -- โš ๏ธ Tags may contain non-genre values (e.g., "Favorites", "To Read") -- โš ๏ธ User filters by "Tags" instead of "Genre" in UI - -**Why acceptable:** -- Calibre has the same "con" and it's not a problem there -- Users can filter their own tags as well (feature, not bug) -- API backward compatibility maintained (genre_filter still works) - ---- - -## ๐Ÿ“‹ Implementation Checklist - -### Phase 1: Backend - SQL Layer (25 minutes) -- [ ] Add `tags_filter` parameter to `SearchMediaItemsUnified` query -- [ ] Add fuzzy matching filter logic with EXISTS clause and word_similarity() -- [ ] Add tag similarity scoring to ORDER BY clause (GREATEST function) -- [ ] Keep `genre_filter` for backward compatibility -- [ ] Regenerate Go code with `sqlc generate` -- [ ] Test SQL query in database console (verify fuzzy matching works) - -### Phase 2: Backend - Service Layer (20 minutes) -- [ ] Add `TagsFilter string` to `SearchParams` struct -- [ ] Update `dbParams` building to include tags_filter -- [ ] Update handler to extract `tags_filter` query parameter -- [ ] Keep existing `genre_filter` extraction (backward compat) - -### Phase 3: Backend - Tags Autocomplete API (30 minutes) -- [ ] Add `SearchTagsValues` SQL query -- [ ] Add "tags" case to `SearchFieldValues` service -- [ ] Add tags route to handler (autocomplete support) -- [ ] Test tags autocomplete endpoint - -### Phase 4: Frontend - TypeScript (20 minutes) -- [ ] Rename `fetchGenreValues()` โ†’ `fetchTagValues()` in bookshelf.ts -- [ ] Update field id from "genre" โ†’ "tags" in custom-section-builder.ts -- [ ] Update collection rules from "genre" โ†’ "tags" in collection-rules.ts -- [ ] No other TS files need changes (search.ts is header search only) - -### Phase 5: Frontend - Templates (15 minutes) -- [ ] Update bookshelf template (genre โ†’ tags labels/inputs) -- [ ] Change datalist id from "genre-datalist" โ†’ "tags-datalist" -- [ ] Update @input handler from `fetchGenreValues` โ†’ `fetchTagValues` -- [ ] Regenerate template Go files with `templ generate` - -### Phase 6: Bruno Collection Updates (15 minutes) -- [ ] Create `Fuzzy Tags Filter.yml` (test fuzzy matching behavior) -- [ ] Create `Field Values Search - Tags.yml` (test autocomplete) -- [ ] Update `Combined Search and Filters.yml` (genre โ†’ tags_filter) -- [ ] Add test cases for fuzzy matching (e.g., "Sci Fi" โ†’ "Science Fiction") -- [ ] Add documentation for fuzzy tags filter behavior - -### Phase 7: Documentation (20 minutes) -- [ ] Create `docs/developer/api/media-items/search_media_items.md` (or update existing) -- [ ] Document tags_filter parameter -- [ ] Document backward compatibility with genre_filter -- [ ] Add examples of tags filtering -- [ ] Update any relevant user documentation - -### Phase 8: Integration Tests (30 minutes) -- [ ] Create `tags_filter_test.go` in `cmd/server/tests/` -- [ ] Test with `setupTestServer()` helper from `test_helpers_test.go` -- [ ] Test tags filter returns matching books -- [ ] Test tags autocomplete endpoint -- [ ] Test backward compatibility (genre_filter still works) -- [ ] Test combined filters (tags + author + series) - -### Phase 9: Verification & Testing (20 minutes) -- [ ] Run full test suite: `go test ./... -v` -- [ ] Test tags filter in browser (manual QA) -- [ ] Verify tags autocomplete works -- [ ] Verify backward compatibility (genre_filter) -- [ ] Check no regressions in existing filters - ---- - -## ๐Ÿ”ง Technical Changes - -### 1. SQL Query Changes - -**File:** `internal/database/queries/queries.sql` - -**Location:** Line 435-437 (SearchMediaItemsUnified query) - -**Change:** -```sql --- BEFORE: --- Fuzzy genre filter -AND (sqlc.narg('genre_filter') = '' OR word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')) > 0.3) - --- AFTER: --- Fuzzy genre filter (kept for backward compatibility) -AND (sqlc.narg('genre_filter') = '' OR word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')) > 0.3) --- Tags filter (NEW - fuzzy match against tags array) -AND (sqlc.narg('tags_filter') = '' OR EXISTS ( - SELECT 1 FROM unnest(mi.tags_search) AS tag - WHERE word_similarity(sqlc.narg('tags_filter'), tag) > 0.3 -)) -``` - -**New Query:** Add `SearchTagsValues` for autocomplete -```sql --- name: SearchTagsValues :many -SELECT - unnest(mi.tags_search) as value, - COUNT(*) as count, - word_similarity(sqlc.narg('search_query'), unnest(mi.tags_search))::float8 as score -FROM media_items mi -JOIN libraries l ON mi.library_id = l.id -LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id') -WHERE COALESCE(lv.is_visible, true) = true - AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid) - AND unnest(mi.tags_search) IS NOT NULL - AND word_similarity(sqlc.narg('search_query'), unnest(mi.tags_search)) > 0.3 -GROUP BY unnest(mi.tags_search), word_similarity(sqlc.narg('search_query'), unnest(mi.tags_search))::float8 -ORDER BY word_similarity(sqlc.narg('search_query'), unnest(mi.tags_search))::float8 DESC, COUNT DESC -LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset'); -``` - -**ORDER BY Clause Update (line ~484):** -Add tag similarity scoring to the GREATEST() calculation: -```sql --- In the ORDER BY clause, add to the GREATEST() function: -GREATEST( - CASE WHEN sqlc.narg('is_exact_search') = false THEN - word_similarity(sqlc.narg('search_query'), mi.title) - ELSE 0 END, - CASE WHEN sqlc.narg('is_exact_search') = false THEN - word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, '')) - ELSE 0 END, - word_similarity(sqlc.narg('author_filter'), COALESCE(mi.author, '')), - word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')), - (SELECT MAX(word_similarity(sqlc.narg('tags_filter'), tag)) - FROM unnest(mi.tags_search) AS tag) -- NEW: Tag similarity score -) -``` - -This ensures results with better tag matches appear first. - -### 2. Service Layer Changes - -**File:** `internal/services/search.go` - -**Change 1:** Update SearchParams struct (line 22-36) -```go -type SearchParams struct { - UserID pgtype.UUID - LibraryID pgtype.UUID - AuthorFilter string - SeriesFilter string - GenreFilter string // KEPT for backward compatibility - TagsFilter string // NEW - LanguageFilter string - YearMin int - YearMax int - HasCover bool - SearchQuery string - Sort string - Limit int - Offset int -} -``` - -**Change 2:** Update dbParams building (line 67-82) -```go -dbParams := database.SearchMediaItemsUnifiedParams{ - UserID: params.UserID, - LibraryID: params.LibraryID, - AuthorFilter: pgtype.Text{String: params.AuthorFilter, Valid: params.AuthorFilter != ""}, - SeriesFilter: pgtype.Text{String: params.SeriesFilter, Valid: params.SeriesFilter != ""}, - GenreFilter: pgtype.Text{String: params.GenreFilter, Valid: params.GenreFilter != ""}, // KEPT - TagsFilter: pgtype.Text{String: params.TagsFilter, Valid: params.TagsFilter != ""}, // NEW - LanguageFilter: pgtype.Text{String: params.LanguageFilter, Valid: params.LanguageFilter != ""}, - // ... rest of params -} -``` - -**Change 3:** Add tags case to SearchFieldValues (after line 151) -```go -case "tags": - results, err := s.db.SearchTagsValues(ctx, database.SearchTagsValuesParams{ - SearchQuery: pgtype.Text{String: params.SearchQuery, Valid: true}, - UserID: params.UserID, - LibraryID: params.LibraryID, - Limit: pgtype.Int4{Int32: int32(params.Limit), Valid: true}, - Offset: pgtype.Int4{Int32: int32(params.Offset), Valid: true}, - }) - if err != nil { - return nil, err - } - fieldValues := make([]FieldValue, len(results)) - for i, r := range results { - fieldValues[i] = FieldValue{Value: r.Value.String, Count: r.Count, Score: r.Score} - } - return fieldValues, nil -``` - -### 3. Handler Changes - -**File:** `internal/handlers/media.go` - -**Change 1:** Extract tags_filter parameter (after line 1414) -```go -// Extract filter parameters -authorFilter := c.QueryParam("author_filter") -seriesFilter := c.QueryParam("series_filter") -genreFilter := c.QueryParam("genre_filter") // KEPT for backward compatibility -tagsFilter := c.QueryParam("tags_filter") // NEW -languageFilter := c.QueryParam("language_filter") -``` - -**Change 2:** Add to SearchParams (line 1426-1440) -```go -params := services.SearchParams{ - UserID: userID.ID, - LibraryID: libUUID, - SearchQuery: query, - AuthorFilter: authorFilter, - SeriesFilter: seriesFilter, - GenreFilter: genreFilter, // KEPT - TagsFilter: tagsFilter, // NEW - LanguageFilter: languageFilter, - // ... rest of params -} -``` - -**Change 3:** Add tags autocomplete route (after line 1379) -```go -if tags := c.QueryParam("tags"); tags != "" { - return mh.handleFieldValuesSearch(c, userID.ID, "tags", tags) -} -``` - -### 4. Frontend TypeScript Changes - -**File:** `web/src/bookshelf.ts` - -**Change:** Line 297-300 -```typescript -// AFTER (add new function, keep old commented out): -async fetchTagValues(input: HTMLInputElement): Promise { - await this.fetchFieldValues("tags", input.value, "tags-datalist"); -} - -// TODO: Uncomment if genre field is populated in future -// async fetchGenreValues(input: HTMLInputElement): Promise { -// await this.fetchFieldValues("genre", input.value, "genre-datalist"); -// } -``` - -**File:** `web/src/custom-section-builder.ts` - -**Change:** Line 49-50 -```typescript -// AFTER (add new field, keep old commented out): -[ - { id: "tags", label: "Tags" }, - // TODO: Uncomment if genre field is populated in future - // { id: "genre", label: "Genre" }, - // ... other fields -] -``` - -### 5. Template Changes - -**File:** `templates/bookshelf.templ` (source file) - -**Locate:** Find the genre filter input section (around line 131 in generated Go file) - -**Change:** -```html - - -
- - - -
- - -{{- /* Genre Filter with Autocomplete */} -{{- /*
*/} -{{- /* */} -{{- /* */}} -{{- /* */}} -{{- /*
*/}} -``` - -**Then regenerate:** `templ generate` - -**Note:** The genre filter code is preserved in the template as a comment, making it easy to re-enable in the future if the genre field is populated. - ---- - -## ๐Ÿงช Testing Strategy - -### Integration Tests - -**File:** `cmd/server/tests/tags_filter_test.go` (NEW) - -**Test Structure:** -```go -package main - -import ( - "testing" - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -func TestTagsFilter(t *testing.T) { - setup := setupDeviceTest(t) // โœ… Using test_helpers - defer setup.Server.Close() - - libraryID := setup.CreateLibrary(t, "Test Tags Library", "ebooks") - _ = setup.CreateDevice(t, "Test Device", "koreader", "tags-test-123") - - client := &http.Client{} - - // Add folder to library - addFolderToLibrary(t, setup, libraryID, "/app/uploads") - - // Helper to create book with tags - createBook := func(title, author string, tags []string) { - bookReq := map[string]interface{}{ - "library_id": libraryID, - "title": title, - "author": author, - "tags": tags, - "file_path": "/tmp/test.epub", - "file_size": 1024, - "mime_type": "application/epub+zip", - } - // ... create book - } - - // Create test books with different tags - createBook("Foundation", "Asimov", []string{"Science Fiction", "Adventure"}) - createBook("Dune", "Herbert", []string{"Science Fiction", "Dystopian"}) - createBook("Neuromancer", "Gibson", []string{"Cyberpunk", "Science Fiction"}) - createBook("The Hobbit", "Tolkien", []string{"Fantasy", "Adventure"}) - - t.Run("Tags filter - Science Fiction", func(t *testing.T) { - req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&tags_filter=Science+Fiction", nil) - req.Header.Set("Authorization", "Bearer "+setup.UserToken) - // ... test implementation - // Should return Foundation, Dune, Neuromancer (all have "Science Fiction" tag) - }) - - t.Run("Tags filter - fuzzy match", func(t *testing.T) { - req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&tags_filter=Sci+Fi", nil) - req.Header.Set("Authorization", "Bearer "+setup.UserToken) - // ... test implementation - // Should return Foundation, Dune, Neuromancer (fuzzy matches "Science Fiction") - // Results should be ranked by similarity score - }) - - t.Run("Tags filter - Fantasy", func(t *testing.T) { - // Should return only The Hobbit - }) - - t.Run("Tags autocomplete", func(t *testing.T) { - req := httptest.NewRequest("GET", "/api/media-items/search?tags=Sci&library_id="+libraryID, nil) - req.Header.Set("Authorization", "Bearer "+setup.UserToken) - // Should return "Science Fiction" in autocomplete results with fuzzy matching - }) - - t.Run("Backward compatibility - genre_filter still works", func(t *testing.T) { - // Verify genre_filter doesn't break (returns 0 results on NULL genre) - }) - - t.Run("Combined filters - tags + author", func(t *testing.T) { - // tags_filter=Sci+Fi&author_filter=Asimov - // Should return only Foundation - }) -} -``` - -### Manual QA Checklist - -- [ ] Tags filter returns books with matching tags -- [ ] Tags filter uses fuzzy matching (e.g., "Sci Fi" matches "Science Fiction") -- [ ] Results are ranked by tag similarity score (best matches first) -- [ ] Tags autocomplete shows tag values as you type -- [ ] Multiple tags in book are all searchable -- [ ] Combined filters work (tags + author + series) -- [ ] Empty tags_filter returns all books (no filter) -- [ ] Backward compatibility: genre_filter still works (returns 0 on NULL) -- [ ] Browser console shows no errors -- [ ] HTMX requests complete successfully -- [ ] UI updates without page reload - ---- - -## ๐Ÿ“ Documentation - -### API Documentation - -**File:** `docs/developer/api/media-items/search_media_items.md` (CREATE or UPDATE) - -**Content:** -```markdown -# Search Media Items - -## GET /api/media-items/search - -Search and filter media items with fuzzy matching and tag-based filtering. - -### Query Parameters - -| Parameter | Type | Required | Description | -|-----------|------|----------|-------------| -| `q` | string | No | Search query (matches title, author, series, tags) | -| `library_id` | string | No | Filter by library UUID | -| `author_filter` | string | No | Fuzzy match author field | -| `series_filter` | string | No | Fuzzy match series field | -| `tags_filter` | string | No | **Fuzzy match tags array** (NEW) | -| `language_filter` | string | No | Fuzzy match language field | -| `year_min` | integer | No | Minimum copyright year | -| `year_max` | integer | No | Maximum copyright year | -| `has_cover` | boolean | No | Filter by cover existence | -| `sort` | string | No | Sort order (e.g., "title ASC") | -| `limit` | integer | No | Max results (default: 50) | -| `offset` | integer | No | Results offset (default: 0) | - -### Tags Filter Behavior - -**New in v1.x:** The `tags_filter` parameter filters books by their tags array. - -- **Matching:** Fuzzy match using `word_similarity()` - matches tags with similarity > 0.3 -- **Calibre Integration:** Works seamlessly with Calibre tags (which include genre) -- **Result Ranking:** Best tag matches appear first in results -- **Example Tags:** `["Science Fiction", "Adventure", "Dystopian"]` - -**Fuzzy Matching Examples:** - -```bash -# Exact match: "Science Fiction" -GET /api/media-items/search?tags_filter=Science+Fiction -# Returns: All books with "Science Fiction" tag - -# Fuzzy match: "Sci Fi" โ†’ "Science Fiction" -GET /api/media-items/search?tags_filter=Sci+Fi -# Returns: Books with "Science Fiction", "Sci-Fi", "Scifi" (score > 0.3) - -# Combine tags with author -GET /api/media-items/search?tags_filter=Sci+Fi&author_filter=Asimov -# Returns: Science Fiction books by Asimov, best matches first - -# Fuzzy match with variations -GET /api/media-items/search?tags_filter=Cyber -# Returns: Books with "Cyberpunk", "Cyber", "Cyberthriller" tags -``` - -### Backward Compatibility - -The `genre_filter` parameter is **deprecated but still supported** for backward compatibility. It will return 0 results for books imported from Calibre (genre field is NULL). - -**Migration Guide:** -```javascript -// OLD (deprecated): -genre_filter=Science+Fiction - -// NEW (recommended): -tags_filter=Science+Fiction -``` - -### Response - -**Status Codes:** -- `200 OK` - Results found -- `404 Not Found` - No results match filters -- `401 Unauthorized` - Missing or invalid JWT -- `500 Internal Server Error` - Server error - -**Response Body:** -```json -[ - { - "id": "uuid", - "title": "Foundation", - "author": "Isaac Asimov", - "tags": ["Science Fiction", "Adventure"], - "library_id": "uuid", - "library_name": "E-Books" - } -] -``` - -### User Documentation - -**File:** `docs/user/searching-and-filtering.md` (UPDATE or CREATE) - -**Content:** -```markdown -# Searching and Filtering Books - -## Tag-Based Filtering - -Bookhoard uses Calibre's tag system for categorization. When you import books from Calibre, their tags are automatically available for filtering. - -### How Tags Work - -Tags are keywords or categories assigned to books, such as: -- **Genres:** Science Fiction, Fantasy, Mystery, Romance -- **Categories:** Favorites, To Read, Read -- **Metadata:** Ebook, Owned, Borrowed - -### Filtering by Tags - -1. Navigate to the **Bookshelf** page -2. Use the **Tags** filter input -3. Start typing to see autocomplete suggestions -4. Select a tag or press Enter to filter - -**Example:** Typing "Sci" will suggest "Science Fiction" - -### Combining Filters - -You can combine tags with other filters: -- **Tags + Author:** Find all Science Fiction by Asimov -- **Tags + Series:** Find all Fantasy in the "Wheel of Time" series -- **Tags + Year:** Find all Dystopian fiction published after 2000 - -### Tips - -- Tags from Calibre are automatically imported -- You can add custom tags when editing book metadata -- Tags use fuzzy matching - "Sci Fi" will match "Science Fiction" -- Best matching tags appear first in results -- Autocomplete shows existing tags as you type with relevance scores -``` - ---- - -## ๐ŸŽฏ Git Commit Structure - -### Commit 1: Backend SQL Layer -``` -feat: add fuzzy tags_filter to search query - -- Add fuzzy tags_filter parameter to SearchMediaItemsUnified -- Add EXISTS clause with word_similarity() for fuzzy tag matching -- Add tag similarity scoring to ORDER BY clause -- Add SearchTagsValues query for autocomplete -- Keep genre_filter for backward compatibility - -Relates to # -``` - -### Commit 2: Backend Service Layer -``` -feat: implement tags filter in service layer - -- Add TagsFilter to SearchParams struct -- Update dbParams building to include tags_filter -- Add tags case to SearchFieldValues service - -Relates to # -``` - -### Commit 3: Backend Handler Layer -``` -feat: add tags filter and autocomplete endpoints - -- Extract tags_filter query parameter in handler -- Add tags autocomplete route -- Keep genre_filter for backward compatibility - -Relates to # -``` - -### Commit 4: Frontend TypeScript -``` -refactor: replace genre with tags in frontend TypeScript - -- Add fetchTagValues function (keep fetchGenreValues commented out) -- Update custom-section-builder field id to "tags" (keep "genre" commented) -- Update collection-rules field mappings to "tags" (keep "genre" commented) -- Genre code preserved for easy restoration if field is populated later - -Relates to # -``` - -### Commit 5: Frontend Templates -``` -refactor: replace genre with tags in bookshelf UI - -- Add Tags filter input (keep Genre input commented out) -- Update input name to tags_filter (keep genre_filter in comment) -- Update datalist and Alpine.js bindings -- Genre HTML preserved in template comments for future use - -Relates to # -``` - -### Commit 6: Bruno Collection -``` -test: update Bruno requests for tags filter - -- Add Fuzzy Tags Filter.yml -- Add Field Values Search - Tags.yml -- Update Combined Search and Filters.yml - -Relates to # -``` - -### Commit 7: Documentation -``` -docs: document tags filter API and usage - -- Add search_media_items API documentation -- Add user guide for tag-based filtering -- Document backward compatibility with genre_filter - -Relates to # -``` - -### Commit 8: Integration Tests -``` -test: add integration tests for tags filter - -- Create tags_filter_test.go -- Test tags filter functionality -- Test tags autocomplete endpoint -- Test backward compatibility -- Test combined filters - -Uses test_helpers.setupTestServer() -Relates to # -``` - ---- - -## โœ… Verification Checklist - -### Before Completing Task - -- [ ] All code compiles: `go build ./...` -- [ ] All tests pass: `go test ./... -v` -- [ ] Guidelines verified: `bash scripts/verify-guidelines.sh` (0 errors) -- [ ] Documentation renders at `/docs` endpoint -- [ ] Bruno requests execute successfully -- [ ] Manual QA completed in browser -- [ ] No regressions in existing filters -- [ ] Git commits are logical and well-structured -- [ ] Only intended changes in `git diff` - -### Post-Deployment Verification - -- [ ] Tags filter works in production -- [ ] Tags autocomplete returns results -- [ ] Backward compatibility maintained (genre_filter) -- [ ] No performance issues -- [ ] User feedback collected - ---- - -## ๐Ÿ“š References - -- **Project Guidelines:** `PROJECT_GUIDELINES.md` -- **Service Layer Pattern:** `internal/services/` -- **Test Helpers:** `cmd/server/tests/test_helpers_test.go` -- **Bruno Collection:** `bruno/media-items/` -- **API Documentation:** `docs/developer/api/` -- **User Documentation:** `docs/user/` - ---- - -## ๐Ÿš€ Next Steps - -1. **Review this plan** and approve approach -2. **Implement Phase 1-3** (Backend changes) -3. **Implement Phase 4-5** (Frontend changes) -4. **Implement Phase 6-7** (Bruno + Docs) -5. **Implement Phase 8-9** (Tests + Verification) -6. **Create git commits** following commit structure -7. **Push changes** and verify in production - ---- - -**Last Updated:** 2026-03-25 -**Status:** Ready for implementation diff --git a/UNIFIED_SEARCH_IMPLEMENTATION.md b/UNIFIED_SEARCH_IMPLEMENTATION.md deleted file mode 100644 index db4a464..0000000 --- a/UNIFIED_SEARCH_IMPLEMENTATION.md +++ /dev/null @@ -1,1924 +0,0 @@ -# Unified Search and Filter Implementation Plan - -## Executive Summary - -**Goal:** Consolidate `/api/media-items/filtered` and `/api/media-items/search` endpoints into a single unified `/api/media-items/search` endpoint that supports: -- All-fuzzy filters (except years/booleans) -- Google-style "exact match in quotes" for search queries -- Field-specific fuzzy search for autocomplete dropdowns -- Combined search + filters functionality -- Backward compatibility with saved filters - -**Approach:** Surgical, incremental changes that reuse existing code, following PROJECT_GUIDELINES.md strictly. - -**Technical Note - sqlc v1.30.0 Limitation:** -Autocomplete dropdowns use 4 separate simple queries (one per field type) instead of 1 complex query due to sqlc v1.30.0's inability to parse complex CASE expressions in GROUP BY clauses. This approach is simpler, works correctly with the current sqlc version, and the service layer routes to the appropriate query based on field type. - ---- - -## Implementation Status (as of March 23, 2026) - -### โœ… COMPLETED -- **Phase 1:** Database Schema - GIN indexes for pg_trgm fuzzy search -- **Phase 2:** SQL Queries - SearchMediaItemsUnified + 4 field value queries -- **Phase 3:** Search Service - SearchMediaItemsUnified() + SearchFieldValues() methods -- **Phase 8:** No Changes Needed - Correctly follows established patterns - -### โš ๏ธ PARTIAL (Ready to Implement) -- **Phase 4.1:** Backend Handler - Code specified in plan, ready to implement - - โœ… Complete handler code with autocomplete detection - - โœ… Complete handleFieldValuesSearch() method - - โœ… Fixed QueryParam bug (echo doesn't support default values) - - โณ Ready to paste into media.go - -- **Phase 4.2-4.3:** Frontend Templates - Updated in plan with search button approach - - โœ… Search button + Enter key triggers (no blur trigger) - - โœ… Pure HTML5 datalist (no Alpine.js state) - - โœ… All filter inputs with autocomplete - - โณ Ready to paste into bookshelf.templ - -- **Phase 4.4:** Frontend TypeScript - Specified in plan - - โœ… fetchFieldValues() function with native DOM manipulation - - โœ… fetchAuthorValues(), fetchGenreValues(), etc. helper functions - - โœ… Fixed query param names (singular: author, genre, etc.) - - โณ Ready to add to bookshelf.ts - -### โŒ NOT STARTED -- **Phase 5:** Frontend Build Verification -- **Phase 6:** Tests - search_unified_test.go needs to be created -- **Phase 7:** Documentation Updates - docs/developer/api/media-items/search_media_items.md needs update -- **Phase 9:** Cleanup - Remove deprecated /filtered endpoint after testing -- **Phase 10:** Verification - Manual testing of all features - -### ๐Ÿ“‹ IMPLEMENTATION ORDER (Recommended) -1. **Phase 4.1** - Backend Handler (media.go) - ~30 min -2. **Phase 4.2-4.3** - Frontend Templates (bookshelf.templ) - ~20 min -3. **Phase 4.4** - Frontend TypeScript (bookshelf.ts) - ~15 min -4. **Phase 5** - Build & Verify (npm run build, go build) - ~5 min -5. **Phase 6** - Manual Testing (autocomplete, filters, search) - ~15 min -6. **Phase 7** - Tests (create search_unified_test.go) - ~1 hour -7. **Phase 8** - Documentation updates - ~30 min -8. **Phase 9** - Cleanup (remove /filtered) - ~10 min - -**Total remaining: ~3 hours** - ---- - -## Current State Analysis - -### Existing Endpoints - -#### 1. `/api/media-items/search` (internal/router/search.go:11) -- **Purpose:** Global fuzzy search across all libraries -- **Handler:** `SearchMediaItems` (internal/handlers/media.go:1419-1493) -- **Logic:** - - Uses `SearchMediaItems` query (ILIKE pattern matching) - - Falls back to `SearchMediaItemsFuzzy` query (pg_trgm `word_similarity()`) - - Threshold: 0.3 similarity -- **Parameters:** `q`, `library_id`, `limit`, `offset` -- **Used by:** Search box (incorrectly - currently calls `/filtered`) - -#### 2. `/api/media-items/filtered` (internal/router/media.go:17) -- **Purpose:** Exact match filtering -- **Handler:** `ListMediaItemsFiltered` (internal/handlers/media.go:705-766) -- **Logic:** - - `ListMediaItemsFiltered` query (queries.sql:227-268) - - Author/series: `ILIKE` (case-insensitive, NO wildcards) - - Genre/language: `=` (exact match) - - Years: `>=`, `<=` (exact range) - - Boolean: exact match -- **Parameters:** `author_filter`, `series_filter`, `genre_filter`, `language_filter`, `year_min`, `year_max`, `has_cover`, `sort`, `limit`, `offset` -- **Used by:** All filter fields, search box (incorrectly) - -### Current Frontend Implementation - -**Search box** (templates/bookshelf.templ:73-83): -```html - - hx-trigger="keyup changed delay:300ms" - hx-include="#filter-form"> -``` - -**Filter fields** (templates/bookshelf.templ:90-117): -```html - - - -``` - -### Database Schema - -**Indexes exist** (database/schema/schema.sql): -- B-tree indexes: author, genre, language, series, copyright_year (lines 365-373) -- GIN indexes: tags_search, contributors_search (lines 145-146) - -**Missing:** GIN indexes for pg_trgm fuzzy search on text fields - -### Saved Filters - -**Implementation:** Frontend-only (web/src/bookshelf.ts:44-173) -- Stores filter config as JSON in database -- Loads filter config into form via `loadFilter()` method -- Makes API call to `/api/saved-filters/:id` -- Populates form fields with exact values - -**Impact:** Should work seamlessly with new endpoint (only field names matter) - ---- - -## Implementation Phases - -### Phase 1: Database Schema - Add GIN Indexes - -**File:** `database/schema/schema.sql` - -**Add after line 146:** - -```sql --- Add GIN indexes for pg_trgm fuzzy search performance -CREATE INDEX IF NOT EXISTS idx_media_items_author_trgm - ON media_items USING GIN (author gin_trgm_ops); -CREATE INDEX IF NOT EXISTS idx_media_items_title_trgm - ON media_items USING GIN (title gin_trgm_ops); -CREATE INDEX IF NOT EXISTS idx_media_items_series_trgm - ON media_items USING GIN (series gin_trgm_ops); -CREATE INDEX IF NOT EXISTS idx_media_items_genre_trgm - ON media_items USING GIN (genre gin_trgm_ops); -CREATE INDEX IF NOT EXISTS idx_media_items_language_trgm - ON media_items USING GIN (language gin_trgm_ops); -``` - -**Verification:** -```bash -podman compose down -v -podman compose up -d -podman exec bookhoard_db psql -U postgres -d bookhoard -c "\d+ media_items" -``` - ---- - -### Phase 2: SQL Queries - Add Unified Search Query - -**File:** `internal/database/queries/queries.sql` - -**Add after `SearchMediaItemsFuzzy` (line 460):** - -```sql --- name: SearchMediaItemsUnified :many -SELECT mi.*, l.name as library_name, lt.name as library_type_name -FROM media_items mi -JOIN libraries l ON mi.library_id = l.id -JOIN library_types lt ON l.library_type_id = lt.id -LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id') -WHERE COALESCE(lv.is_visible, true) = true - AND mi.library_id = sqlc.narg('library_id') - -- Fuzzy author filter - AND (sqlc.narg('author_filter') = '' OR word_similarity(sqlc.narg('author_filter'), COALESCE(mi.author, '')) > 0.3) - -- Fuzzy series filter - AND (sqlc.narg('series_filter') = '' OR word_similarity(sqlc.narg('series_filter'), COALESCE(mi.series, '')) > 0.3) - -- Fuzzy genre filter - AND (sqlc.narg('genre_filter') = '' OR word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')) > 0.3) - -- Fuzzy language filter - AND (sqlc.narg('language_filter') = '' OR word_similarity(sqlc.narg('language_filter'), COALESCE(mi.language, '')) > 0.3) - -- Year range (exact) - AND (sqlc.narg('year_min') = 0 OR mi.copyright_year >= sqlc.narg('year_min')) - AND (sqlc.narg('year_max') = 0 OR mi.copyright_year <= sqlc.narg('year_max')) - -- Boolean (exact) - AND (sqlc.narg('has_cover') = false OR mi.cover_image_path IS NOT NULL) - -- Search query (fuzzy or exact based on quotes) - AND ( - sqlc.narg('search_query') = '' OR - -- Fuzzy search (default) - sqlc.narg('is_exact_search') = false AND ( - word_similarity(sqlc.narg('search_query'), mi.title) > 0.3 OR - word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, '')) > 0.3 OR - word_similarity(sqlc.narg('search_query'), COALESCE(mi.series, '')) > 0.3 OR - EXISTS ( - SELECT 1 FROM unnest(mi.tags_search) AS tag - WHERE word_similarity(sqlc.narg('search_query'), tag) > 0.3 - LIMIT 1 - ) OR - EXISTS ( - SELECT 1 FROM unnest(mi.contributors_search) AS contributor - WHERE word_similarity(sqlc.narg('search_query'), contributor) > 0.3 - LIMIT 1 - ) - ) OR - -- Exact search (with quotes) - sqlc.narg('is_exact_search') = true AND ( - mi.title ILIKE sqlc.narg('search_pattern') OR - mi.author ILIKE sqlc.narg('search_pattern') OR - mi.series ILIKE sqlc.narg('search_pattern') OR - sqlc.narg('search_pattern') = ANY(mi.tags_search) OR - sqlc.narg('search_pattern') = ANY(mi.contributors_search) - ) - ) -ORDER BY - CASE - WHEN sqlc.narg('search_query') != '' THEN - GREATEST( - CASE WHEN sqlc.narg('is_exact_search') = false THEN - word_similarity(sqlc.narg('search_query'), mi.title) - ELSE 0 END, - CASE WHEN sqlc.narg('is_exact_search') = false THEN - word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, '')) - ELSE 0 END, - word_similarity(sqlc.narg('author_filter'), COALESCE(mi.author, '')), - word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')) - ) - ELSE 0 - END DESC, - mi.title ASC -LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset'); -``` - -**Add 4 separate field value search queries (for autocomplete dropdowns):** - -**Note:** Using 4 separate simple queries instead of 1 complex query due to sqlc v1.30.0 limitation with CASE expressions in GROUP BY clauses. This approach is simpler and works correctly with the current sqlc version. - -```sql --- name: SearchAuthorValues :many -SELECT - mi.author as value, - COUNT(*) as count, - word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, ''))::float8 as score -FROM media_items mi -JOIN libraries l ON mi.library_id = l.id -LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id') -WHERE COALESCE(lv.is_visible, true) = true - AND mi.library_id = sqlc.narg('library_id') - AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, '')) > 0.3 - AND mi.author IS NOT NULL - AND mi.author != '' -GROUP BY mi.author -ORDER BY score DESC, count DESC -LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset'); - --- name: SearchGenreValues :many -SELECT - mi.genre as value, - COUNT(*) as count, - word_similarity(sqlc.narg('search_query'), COALESCE(mi.genre, ''))::float8 as score -FROM media_items mi -JOIN libraries l ON mi.library_id = l.id -LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id') -WHERE COALESCE(lv.is_visible, true) = true - AND mi.library_id = sqlc.narg('library_id') - AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.genre, '')) > 0.3 - AND mi.genre IS NOT NULL - AND mi.genre != '' -GROUP BY mi.genre -ORDER BY score DESC, count DESC -LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset'); - --- name: SearchSeriesValues :many -SELECT - mi.series as value, - COUNT(*) as count, - word_similarity(sqlc.narg('search_query'), COALESCE(mi.series, ''))::float8 as score -FROM media_items mi -JOIN libraries l ON mi.library_id = l.id -LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id') -WHERE COALESCE(lv.is_visible, true) = true - AND mi.library_id = sqlc.narg('library_id') - AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.series, '')) > 0.3 - AND mi.series IS NOT NULL - AND mi.series != '' -GROUP BY mi.series -ORDER BY score DESC, count DESC -LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset'); - --- name: SearchLanguageValues :many -SELECT - mi.language as value, - COUNT(*) as count, - word_similarity(sqlc.narg('search_query'), COALESCE(mi.language, ''))::float8 as score -FROM media_items mi -JOIN libraries l ON mi.library_id = l.id -LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id') -WHERE COALESCE(lv.is_visible, true) = true - AND mi.library_id = sqlc.narg('library_id') - AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.language, '')) > 0.3 - AND mi.language IS NOT NULL - AND mi.language != '' -GROUP BY mi.language -ORDER BY score DESC, count DESC -LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset'); -``` - -**Regenerate Go code:** -```bash -sqlc generate -``` - -**Verify:** Check `internal/database/queries.sql.go` for new functions: `SearchAuthorValues`, `SearchGenreValues`, `SearchSeriesValues`, `SearchLanguageValues` - ---- - -### Phase 3: Create Search Service - -**File:** `internal/services/search.go` (NEW) - -```go -package services - -import ( - "bookhoard/internal/database" - "context" - "strconv" - "strings" - - "github.com/google/uuid" - "github.com/jackc/pgx/v5/pgtype" -) - -// SearchService handles all search and filter operations -type SearchService struct { - db *database.Queries -} - -// NewSearchService creates a new search service instance -func NewSearchService(db *database.Queries) *SearchService { - return &SearchService{db: db} -} - -// SearchParams contains parameters for unified search -type SearchParams struct { - UserID pgtype.UUID - LibraryID pgtype.UUID - AuthorFilter string - SeriesFilter string - GenreFilter string - LanguageFilter string - YearMin int - YearMax int - HasCover bool - SearchQuery string - Limit int - Offset int -} - -// parseSearchQuery detects quoted strings for exact match search -// Returns (isExact, processedQuery) -// Examples: -// "asimov" โ†’ (false, "asimov") -// "\"Asimov, Isaac\"" โ†’ (true, "Asimov, Isaac") -func (s *SearchService) parseSearchQuery(query string) (bool, string) { - query = strings.TrimSpace(query) - if strings.HasPrefix(query, "\"") && strings.HasSuffix(query, "\"") && len(query) >= 2 { - return true, strings.Trim(query, "\"") - } - return false, query -} - -// SearchMediaItemsUnified handles combined search + filters -// Supports: -// - Fuzzy text filters (author, series, genre, language) -// - Exact filters (year range, has_cover boolean) -// - Fuzzy search query (or exact match with quotes) -// - Combined search + filters -func (s *SearchService) SearchMediaItemsUnified(ctx context.Context, params SearchParams) ([]database.SearchMediaItemsUnifiedRow, error) { - // Parse search query for exact match detection - isExact, searchQuery := s.parseSearchQuery(params.SearchQuery) - searchPattern := "" - if isExact { - searchPattern = "%" + searchQuery + "%" - } - - // Build database parameters - dbParams := database.SearchMediaItemsUnifiedParams{ - UserID: params.UserID, - LibraryID: params.LibraryID, - AuthorFilter: pgtype.Text{String: params.AuthorFilter, Valid: true}, - SeriesFilter: pgtype.Text{String: params.SeriesFilter, Valid: true}, - GenreFilter: pgtype.Text{String: params.GenreFilter, Valid: true}, - LanguageFilter: pgtype.Text{String: params.LanguageFilter, Valid: true}, - YearMin: pgtype.Int4{Int32: int32(params.YearMin), Valid: true}, - YearMax: pgtype.Int4{Int32: int32(params.YearMax), Valid: true}, - HasCover: pgtype.Bool{Bool: params.HasCover, Valid: true}, - SearchQuery: pgtype.Text{String: searchQuery, Valid: true}, - IsExactSearch: pgtype.Bool{Bool: isExact, Valid: true}, - SearchPattern: pgtype.Text{String: searchPattern, Valid: isExact}, - Limit: pgtype.Int4{Int32: int32(params.Limit), Valid: true}, - Offset: pgtype.Int4{Int32: int32(params.Offset), Valid: true}, - } - - // Execute unified search query - results, err := s.db.SearchMediaItemsUnified(ctx, dbParams) - if err != nil { - return nil, err - } - - return results, nil -} - -// FieldSearchParams contains parameters for field-specific search (autocomplete) -type FieldSearchParams struct { - UserID pgtype.UUID - LibraryID pgtype.UUID - FieldType string // "author", "genre", "series", "language" - SearchQuery string - Limit int - Offset int -} - -// FieldValue represents a single field value with metadata -type FieldValue struct { - Value string - Count int64 - Score float64 -} - -// SearchFieldValues handles field-specific search for autocomplete dropdowns -// Returns distinct values with counts and similarity scores -// Uses 4 separate queries (one per field type) for sqlc v1.30.0 compatibility -func (s *SearchService) SearchFieldValues(ctx context.Context, params FieldSearchParams) ([]FieldValue, error) { - switch params.FieldType { - case "author": - results, err := s.db.SearchAuthorValues(ctx, database.SearchAuthorValuesParams{ - SearchQuery: params.SearchQuery, - UserID: params.UserID, - LibraryID: params.LibraryID, - Limit: int32(params.Limit), - Offset: int32(params.Offset), - }) - if err != nil { - return nil, err - } - fieldValues := make([]FieldValue, len(results)) - for i, r := range results { - fieldValues[i] = FieldValue{Value: r.Value, Count: r.Count, Score: r.Score} - } - return fieldValues, nil - - case "genre": - results, err := s.db.SearchGenreValues(ctx, database.SearchGenreValuesParams{ - SearchQuery: params.SearchQuery, - UserID: params.UserID, - LibraryID: params.LibraryID, - Limit: int32(params.Limit), - Offset: int32(params.Offset), - }) - if err != nil { - return nil, err - } - fieldValues := make([]FieldValue, len(results)) - for i, r := range results { - fieldValues[i] = FieldValue{Value: r.Value, Count: r.Count, Score: r.Score} - } - return fieldValues, nil - - case "series": - results, err := s.db.SearchSeriesValues(ctx, database.SearchSeriesValuesParams{ - SearchQuery: params.SearchQuery, - UserID: params.UserID, - LibraryID: params.LibraryID, - Limit: int32(params.Limit), - Offset: int32(params.Offset), - }) - if err != nil { - return nil, err - } - fieldValues := make([]FieldValue, len(results)) - for i, r := range results { - fieldValues[i] = FieldValue{Value: r.Value, Count: r.Count, Score: r.Score} - } - return fieldValues, nil - - case "language": - results, err := s.db.SearchLanguageValues(ctx, database.SearchLanguageValuesParams{ - SearchQuery: params.SearchQuery, - UserID: params.UserID, - LibraryID: params.LibraryID, - Limit: int32(params.Limit), - Offset: int32(params.Offset), - }) - if err != nil { - return nil, err - } - fieldValues := make([]FieldValue, len(results)) - for i, r := range results { - fieldValues[i] = FieldValue{Value: r.Value, Count: r.Count, Score: r.Score} - } - return fieldValues, nil - - default: - return []FieldValue{}, nil - } -} -``` - -**Note:** SearchService is created inside the constructor (not in main.go), following the pattern of `FiltersService` and `CollectionService` which are created inside their respective handlers. - ---- - -### Phase 4: Update Handler to Use Search Service - -**File:** `internal/handlers/media.go` - -#### 4.1 Update SearchMediaItems Handler to Use SearchService - -**Replace the entire `SearchMediaItems` function (lines 1421-1495) with:** - -```go -// SearchMediaItems handles GET /api/media-items/search -// Supports two modes: -// 1. Autocomplete: author=value, genre=value, etc. โ†’ returns field values for dropdowns -// 2. Search: q=value with optional filters โ†’ returns media items -func (mh *MediaHandler) SearchMediaItems(c *echo.Context) error { - // Safely get user from context - userID, ok := c.Get("user").(database.Users) - if !ok { - return c.JSON(http.StatusUnauthorized, map[string]string{"error": "user not authenticated"}) - } - - // Detect autocomplete queries first (author=value, genre=value, etc.) - if author := c.QueryParam("author"); author != "" { - return mh.handleFieldValuesSearch(c, userID.ID, "author", author) - } - if genre := c.QueryParam("genre"); genre != "" { - return mh.handleFieldValuesSearch(c, userID.ID, "genre", genre) - } - if series := c.QueryParam("series"); series != "" { - return mh.handleFieldValuesSearch(c, userID.ID, "series", series) - } - if language := c.QueryParam("language"); language != "" { - return mh.handleFieldValuesSearch(c, userID.ID, "language", language) - } - - // Handle media item search with filters - query := c.QueryParam("q") - libraryID := c.QueryParam("library_id") - - // Validate library_id if provided - var libUUID pgtype.UUID - if libraryID != "" { - lib, err := uuid.Parse(libraryID) - if err != nil { - return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid library_id"}) - } - libUUID = pgtype.UUID{Bytes: lib, Valid: true} - } - - limitStr := c.QueryParam("limit") - limit, _ := strconv.Atoi(limitStr) - if limit == 0 { - limit = 50 - } - - offsetStr := c.QueryParam("offset") - offset, _ := strconv.Atoi(offsetStr) - - // Extract filter parameters - authorFilter := c.QueryParam("author_filter") - seriesFilter := c.QueryParam("series_filter") - genreFilter := c.QueryParam("genre_filter") - languageFilter := c.QueryParam("language_filter") - - yearMinStr := c.QueryParam("year_min") - yearMin, _ := strconv.Atoi(yearMinStr) - - yearMaxStr := c.QueryParam("year_max") - yearMax, _ := strconv.Atoi(yearMaxStr) - - hasCover := c.QueryParam("has_cover") == "true" - - // Build search params - params := services.SearchParams{ - UserID: userID.ID, - LibraryID: libUUID, - SearchQuery: query, - AuthorFilter: authorFilter, - SeriesFilter: seriesFilter, - GenreFilter: genreFilter, - LanguageFilter: languageFilter, - YearMin: yearMin, - YearMax: yearMax, - HasCover: hasCover, - Limit: limit, - Offset: offset, - } - - // Call SearchService instead of DB directly - results, err := mh.searchService.SearchMediaItemsUnified(c.Request().Context(), params) - if err != nil && err != pgx.ErrNoRows { - c.Logger().Error("search error", "error", err.Error()) - return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()}) - } - - if len(results) == 0 { - return c.JSON(http.StatusNotFound, map[string]interface{}{ - "error": "no results found", - "query": query, - "results": []interface{}{}, - }) - } - - return c.JSON(http.StatusOK, results) -} - -// handleFieldValuesSearch handles autocomplete queries (author=value, genre=value, etc.) -// Returns distinct field values with counts and similarity scores for dropdown population -func (mh *MediaHandler) handleFieldValuesSearch(c *echo.Context, userID pgtype.UUID, fieldType, searchQuery string) error { - libraryID := c.QueryParam("library_id") - if libraryID == "" { - return c.JSON(http.StatusBadRequest, map[string]string{"error": "library_id is required"}) - } - - libUUID, err := uuid.Parse(libraryID) - if err != nil { - return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid library_id"}) - } - - limitStr := c.QueryParam("limit") - limit, _ := strconv.Atoi(limitStr) - if limit == 0 { - limit = 50 - } - - offsetStr := c.QueryParam("offset") - offset, _ := strconv.Atoi(offsetStr) - - params := services.FieldSearchParams{ - UserID: userID, - LibraryID: pgtype.UUID{Bytes: libUUID, Valid: true}, - FieldType: fieldType, - SearchQuery: searchQuery, - Limit: limit, - Offset: offset, - } - - results, err := mh.searchService.SearchFieldValues(c.Request().Context(), params) - if err != nil { - c.Logger().Error("field values search error", "error", err.Error(), "fieldType", fieldType) - return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()}) - } - - return c.JSON(http.StatusOK, map[string]interface{}{ - "results": results, - "total": len(results), - }) -} -``` - -#### 4.2 Fix Search Box and Pagination - -**File:** `templates/bookshelf.templ` - -**Update global search box (around line 73-83):** - -```html - - - - - -``` - -**Update pagination buttons (lines 298, 311):** - -```html - -hx-get="/api/media-items/filtered?library_id={ currentLibraryID }&limit={ limit }&offset={ offset - limit }" - - -hx-get="/api/media-items/search?library_id={ currentLibraryID }&limit={ limit }&offset={ offset - limit }" -``` - -**Note:** Change parameter name from `search` to `q` to match backend handler expectation. - -#### 4.3 COMPLETE IMPLEMENTATION - All Fixes in One Section - -**IMPORTANT:** This section contains ALL code changes needed to preserve your sort/save/load features while adding autocomplete. Implement everything in this section only - no need to modify earlier phases. - ---- - -### Step 1: Update SQL Query to Support Sort Parameter - -**File:** `internal/database/queries/queries.sql` - -**Find the `SearchMediaItemsUnified` query (you added this in Phase 2) and replace the ORDER BY clause:** - -**BEFORE:** -```sql -ORDER BY - CASE - WHEN sqlc.narg('search_query') != '' THEN - GREATEST(...) - ELSE 0 - END DESC, - mi.title ASC -``` - -**AFTER:** -```sql -ORDER BY - -- Primary sort: relevance score when searching - CASE - WHEN sqlc.narg('search_query') != '' THEN - GREATEST( - CASE WHEN sqlc.narg('is_exact_search') = false THEN - word_similarity(sqlc.narg('search_query'), mi.title) - ELSE 0 END, - CASE WHEN sqlc.narg('is_exact_search') = false THEN - word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, '')) - ELSE 0 END, - word_similarity(sqlc.narg('author_filter'), COALESCE(mi.author, '')), - word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')) - ) - ELSE 0 - END DESC, - -- Secondary sort: user-specified sort parameter - CASE - WHEN sqlc.narg('sort') = 'title ASC' THEN mi.title - ELSE '' - END ASC, - CASE - WHEN sqlc.narg('sort') = 'title DESC' THEN mi.title - ELSE '' - END DESC, - CASE - WHEN sqlc.narg('sort') = 'author ASC' THEN COALESCE(mi.author, '') - ELSE '' - END ASC, - CASE - WHEN sqlc.narg('sort') = 'author DESC' THEN COALESCE(mi.author, '') - ELSE '' - END DESC, - CASE - WHEN sqlc.narg('sort') = 'created_at ASC' THEN mi.created_at - ELSE '1970-01-01'::timestamp - END ASC, - CASE - WHEN sqlc.narg('sort') = 'created_at DESC' THEN mi.created_at - ELSE '1970-01-01'::timestamp - END DESC, - CASE - WHEN sqlc.narg('sort') = 'page_count ASC' THEN COALESCE(mi.page_count::text, '0') - ELSE '' - END ASC, - CASE - WHEN sqlc.narg('sort') = 'page_count DESC' THEN COALESCE(mi.page_count::text, '0') - ELSE '' - END DESC, - -- Tertiary sort: title (default fallback) - mi.title ASC -``` - -**Regenerate Go code:** -```bash -sqlc generate -``` - ---- - -### Step 2: Update Search Service to Support Sort - -**File:** `internal/services/search.go` - -**Add `Sort` field to `SearchParams` struct:** - -```go -// SearchParams contains parameters for unified search -type SearchParams struct { - UserID pgtype.UUID - LibraryID pgtype.UUID - AuthorFilter string - SeriesFilter string - GenreFilter string - LanguageFilter string - YearMin int - YearMax int - HasCover bool - SearchQuery string - Sort string // ADD THIS LINE - Limit int - Offset int -} -``` - -**Update the database parameters in `SearchMediaItemsUnified` method:** - -```go -// Build database parameters -dbParams := database.SearchMediaItemsUnifiedParams{ - UserID: params.UserID, - LibraryID: params.LibraryID, - AuthorFilter: pgtype.Text{String: params.AuthorFilter, Valid: true}, - SeriesFilter: pgtype.Text{String: params.SeriesFilter, Valid: true}, - GenreFilter: pgtype.Text{String: params.GenreFilter, Valid: true}, - LanguageFilter: pgtype.Text{String: params.LanguageFilter, Valid: true}, - YearMin: pgtype.Int4{Int32: int32(params.YearMin), Valid: true}, - YearMax: pgtype.Int4{Int32: int32(params.YearMax), Valid: true}, - HasCover: pgtype.Bool{Bool: params.HasCover, Valid: true}, - SearchQuery: pgtype.Text{String: searchQuery, Valid: true}, - IsExactSearch: pgtype.Bool{Bool: isExact, Valid: true}, - SearchPattern: pgtype.Text{String: searchPattern, Valid: isExact}, - Sort: pgtype.Text{String: params.Sort, Valid: true}, // ADD THIS LINE - Limit: pgtype.Int4{Int32: int32(params.Limit), Valid: true}, - Offset: pgtype.Int4{Int32: int32(params.Offset), Valid: true}, -} -``` - ---- - -### Step 3: Update Handler to Extract Sort Parameter - -**File:** `internal/handlers/media.go` - -**In the `SearchMediaItems` handler (you added this in Phase 4.1), add sort parameter extraction:** - -**Find this section:** -```go -hasCover := c.QueryParam("has_cover") == "true" - -// Build search params -params := services.SearchParams{ -``` - -**Replace with:** -```go -hasCover := c.QueryParam("has_cover") == "true" - -// Extract sort parameter -sortParam := c.QueryParam("sort") -if sortParam == "" { - sortParam = "title ASC" // Default sort -} - -// Build search params -params := services.SearchParams{ -``` - -**Then add `Sort` to the params struct:** -```go -params := services.SearchParams{ - UserID: userID.ID, - LibraryID: libUUID, - SearchQuery: query, - AuthorFilter: authorFilter, - SeriesFilter: seriesFilter, - GenreFilter: genreFilter, - LanguageFilter: languageFilter, - YearMin: yearMin, - YearMax: yearMax, - HasCover: hasCover, - Sort: sortParam, // ADD THIS LINE - Limit: limit, - Offset: offset, -} -``` - ---- - -### Step 4: Update Template with All Filters + Autocomplete - -**File:** `templates/bookshelf.templ` - -**Replace lines 83-262 (from Search input through end of filter bar) with:** - -```html - -
- - -
- -
- - - -
- -
- - - -
- -
- - - -
- -
- - - -
- -
- -
- - -
-
- -
- -
- -
- - -
- -
- -
- -
- - - -
- -
- -
- - - - -``` - ---- - -### Step 5: Add Autocomplete Functions to TypeScript - -**File:** `web/src/bookshelf.ts` - -**Add these functions to the existing `Alpine.data("bookshelf", () => ({ ... }))` component:** - -```typescript -Alpine.data("bookshelf", () => ({ - // ... existing state and methods ... - - // Autocomplete helper function - async fetchFieldValues( - field: string, - search: string, - datalistId: string - ): Promise { - const token = localStorage.getItem("token"); - if (!token) { - console.error("Not authenticated"); - return; - } - - if (search.length < 2) return; - - const currentLibraryId = ( - document.getElementById("library-select") as HTMLSelectElement - )?.value; - if (!currentLibraryId) { - console.error("No library selected"); - return; - } - - try { - const response = await fetch( - `/api/media-items/search?${field}=${encodeURIComponent( - search - )}&library_id=${currentLibraryId}&limit=50`, - { - headers: { Authorization: `Bearer ${token}` }, - } - ); - - if (!response.ok) { - console.error("Failed to fetch field values"); - return; - } - - const data = await response.json(); - - // Update datalist via DOM manipulation - const datalist = document.getElementById(datalistId); - if (!datalist) { - console.error(`Datalist ${datalistId} not found`); - return; - } - - // Clear existing options - datalist.innerHTML = ""; - - // Add new options - data.results.forEach((item: { value: string; count: number }) => { - const option = document.createElement("option"); - option.value = item.value; - option.textContent = `${item.value} (${item.count})`; - datalist.appendChild(option); - }); - } catch (error) { - console.error("Error fetching field values:", error); - } - }, - - // Fetch author values for autocomplete - async fetchAuthorValues(input: HTMLInputElement): Promise { - await this.fetchFieldValues("author", input.value, "author-datalist"); - }, - - // Fetch genre values for autocomplete - async fetchGenreValues(input: HTMLInputElement): Promise { - await this.fetchFieldValues("genre", input.value, "genre-datalist"); - }, - - // Fetch series values for autocomplete - async fetchSeriesValues(input: HTMLInputElement): Promise { - await this.fetchFieldValues("series", input.value, "series-datalist"); - }, - - // Fetch language values for autocomplete - async fetchLanguageValues(input: HTMLInputElement): Promise { - await this.fetchFieldValues("language", input.value, "language-datalist"); - }, - - // ... rest of existing methods ... -})) -``` - -**Note:** These are regular methods (not reactive), so they won't trigger template re-renders. They just manipulate the DOM directly to populate the `` elements. The functions are available in the template because they're part of the Alpine component scope. - - ---- - -## Summary of Changes - -**What this section does:** -1. โœ… Adds sort parameter support to SQL query (Step 1) -2. โœ… Updates service to pass sort parameter (Step 2) -3. โœ… Updates handler to extract sort parameter (Step 3) -4. โœ… Adds series and language filters with autocomplete (Step 4) -5. โœ… Preserves sort dropdown, save/load buttons, modal (Step 4) -6. โœ… Changes triggers to Enter key instead of instant (Step 4) -7. โœ… Adds TypeScript autocomplete functions using simple pattern (Step 5) - -**What's preserved:** -- โœ… Your sort dropdown (all 5 options) -- โœ… Save Filter button and modal -- โœ… Load Filter button and dropdown -- โœ… Clear Filters button -- โœ… Your existing save/load filter logic - -**What's new:** -- โœ… Series filter with autocomplete -- โœ… Language filter with autocomplete -- โœ… Genre filter autocomplete -- โœ… Sort parameter support in unified search -- โœ… **Simple autocomplete pattern**: Empty `` elements populated via DOM manipulation (no Alpine.js reactive state) - -#### 4.4 NOTE - All Frontend Code is in Section 4.3 - -All TypeScript autocomplete functions are included in **Section 4.3, Step 5** above. - -No additional changes needed to `web/src/bookshelf.ts` beyond what's in Section 4.3. - ---- - -### Phase 6: Tests - -**File:** `cmd/server/tests/search_unified_test.go` (NEW) - -```go -package main - -import ( - "bookhoard/internal/handlers" - "encoding/json" - "net/http" - "net/http/httptest" - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -func TestUnifiedSearch(t *testing.T) { - setup := setupDeviceTest(t) - defer setup.Server.Close() - - libraryID := setup.CreateLibrary(t, "Test Search Library", "ebooks") - _ = setup.CreateDevice(t, "Test Search Device", "koreader", "search-test-123") - - t.Run("Fuzzy author filter", func(t *testing.T) { - req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&author_filter=asimov", nil) - req.Header.Set("Authorization", "Bearer "+setup.UserToken) - rec := httptest.NewRecorder() - setup.Server.Config.Handler.ServeHTTP(rec, req) - - assert.Equal(t, http.StatusOK, rec.Code, "Should fuzzy match author") - }) - - t.Run("Fuzzy genre filter", func(t *testing.T) { - req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&genre_filter=scifi", nil) - req.Header.Set("Authorization", "Bearer "+setup.UserToken) - rec := httptest.NewRecorder() - setup.Server.Config.Handler.ServeHTTP(rec, req) - - assert.Equal(t, http.StatusOK, rec.Code, "Should fuzzy match genre") - }) - - t.Run("Exact match with quotes", func(t *testing.T) { - req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&q=%22Foundation%20and%20Empire%22", nil) - req.Header.Set("Authorization", "Bearer "+setup.UserToken) - rec := httptest.NewRecorder() - setup.Server.Config.Handler.ServeHTTP(rec, req) - - assert.Equal(t, http.StatusOK, rec.Code, "Should exact match quoted query") - }) - - t.Run("Combined search + filters", func(t *testing.T) { - req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&q=foundation&author_filter=asimov", nil) - req.Header.Set("Authorization", "Bearer "+setup.UserToken) - rec := httptest.NewRecorder() - setup.Server.Config.Handler.ServeHTTP(rec, req) - - assert.Equal(t, http.StatusOK, rec.Code, "Should combine search and filters") - }) - - t.Run("Field-specific search for dropdown - authors", func(t *testing.T) { - req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&authors=asimov", nil) - req.Header.Set("Authorization", "Bearer "+setup.UserToken) - rec := httptest.NewRecorder() - setup.Server.Config.Handler.ServeHTTP(rec, req) - - assert.Equal(t, http.StatusOK, rec.Code, "Should return author values") - - var response struct { - Results []struct { - Value string `json:"value"` - Count int64 `json:"count"` - Score float64 `json:"score"` - } `json:"results"` - Total int `json:"total"` - } - err := json.Unmarshal(rec.Body.Bytes(), &response) - require.NoError(t, err, "Should unmarshal field values response") - assert.Greater(t, len(response.Results), 0, "Should have results") - }) - - t.Run("Year range filter (exact)", func(t *testing.T) { - req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&year_min=2000&year_max=2020", nil) - req.Header.Set("Authorization", "Bearer "+setup.UserToken) - rec := httptest.NewRecorder() - setup.Server.Config.Handler.ServeHTTP(rec, req) - - assert.Equal(t, http.StatusOK, rec.Code, "Should filter by year range") - }) - - t.Run("Boolean filter (exact)", func(t *testing.T) { - req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&has_cover=true", nil) - req.Header.Set("Authorization", "Bearer "+setup.UserToken) - rec := httptest.NewRecorder() - setup.Server.Config.Handler.ServeHTTP(rec, req) - - assert.Equal(t, http.StatusOK, rec.Code, "Should filter by has_cover") - }) - - t.Run("Missing library_id", func(t *testing.T) { - req := httptest.NewRequest("GET", "/api/media-items/search?q=test", nil) - req.Header.Set("Authorization", "Bearer "+setup.UserToken) - rec := httptest.NewRecorder() - setup.Server.Config.Handler.ServeHTTP(rec, req) - - assert.Equal(t, http.StatusBadRequest, rec.Code, "Should require library_id") - }) -} -``` - -**Verify tests:** -```bash -go test ./cmd/server/tests -v -run TestUnifiedSearch -``` - ---- - -### Phase 7: Documentation Updates - -#### 6.1 Update API Documentation - -**File:** `docs/developer/api/media-items/search_media_items.md` - -**Replace entire content:** - -```markdown -# Search Media Items (Unified) - -Search and filter media items with fuzzy matching support. - -**Note:** All text filters use fuzzy matching via PostgreSQL pg_trgm (threshold: 0.3 similarity). This handles typos and partial matches automatically. Use quotes for exact match. - -**Endpoint**: `GET /api/media-items/search` -**Auth**: Required - -## Query Parameters - -### Search Parameters - -| Parameter | Type | Required | Description | -| ---------- | ------- | -------- | ---------------------------------------------------- | -| q | string | No | Search query (fuzzy by default, exact in quotes) | -| library_id | string | Yes | Filter to specific library (UUID) | -| limit | integer | No | Number of results (default 50, max 200) | -| offset | integer | No | Number to skip for pagination | - -### Filter Parameters (All Fuzzy Except Years/Booleans) - -| Parameter | Type | Description | -| -------------- | ------- | --------------------------------------------------- | -| author_filter | string | Fuzzy match author field | -| series_filter | string | Fuzzy match series field | -| genre_filter | string | Fuzzy match genre field | -| language_filter| string | Fuzzy match language field | -| year_min | integer | Minimum copyright year (exact range) | -| year_max | integer | Maximum copyright year (exact range) | -| has_cover | boolean | Filter by cover image presence (exact boolean) | - -### Autocomplete Parameters (Field-Specific Search) - -| Parameter | Type | Description | -| ---------- | ------ | ---------------------------------------------- | -| authors | string | Search author values for autocomplete dropdown | -| genres | string | Search genre values for autocomplete dropdown | -| series | string | Search series values for autocomplete dropdown | -| languages | string | Search language values for autocomplete | - -## Request Headers - -| Header | Type | Required | Description | -| ------------- | ------ | -------- | ------------ | -| Authorization | string | Yes | Bearer token | - -## Search Behavior - -### Fuzzy Search (Default) - -Handles typos and partial matches automatically: - -- `"asimov"` โ†’ matches "Asimov, Isaac", "Asimov, Foundation" -- `"scifi"` โ†’ matches "Sci-Fi", "Science Fiction" -- `"azimov"` โ†’ matches "Asimov, Isaac" (typo tolerance) - -### Exact Search (With Quotes) - -Use double quotes for exact phrase matching: - -- `"\"Foundation and Empire\""` โ†’ only "Foundation and Empire" -- `"\"Asimov, Isaac\""` โ†’ only "Asimov, Isaac" - -### Filter Behavior - -**Text filters (fuzzy):** -- `author_filter=asimov` โ†’ fuzzy matches author field -- `genre_filter=scifi` โ†’ fuzzy matches genre field - -**Exact filters:** -- `year_min=2000&year_max=2010` โ†’ exact year range -- `has_cover=true` โ†’ exact boolean match - -## Example Requests - -### 1. Global Fuzzy Search - -Search all fields for "foundation": - -```http -GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&q=foundation -Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... -``` - -### 2. Fuzzy Author Filter - -Find books by "asimov" (matches "Asimov, Isaac"): - -```http -GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&author_filter=asimov -Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... -``` - -### 3. Combined Search + Filters - -Search "foundation" within books by "asimov": - -```http -GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&q=foundation&author_filter=asimov -Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... -``` - -### 4. Exact Match with Quotes - -Exact phrase search: - -```http -GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&q="Foundation%20and%20Empire" -Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... -``` - -### 5. Multiple Fuzzy Filters - -Fiction books from 2000-2010: - -```http -GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&genre_filter=fiction&year_min=2000&year_max=2010 -Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... -``` - -### 6. Field-Specific Search (Autocomplete) - -Get author values for dropdown: - -```http -GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&authors=asimov&limit=50 -Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... -``` - -**Response:** -```json -{ - "results": [ - {"value": "Asimov, Isaac", "count": 47, "score": 0.8}, - {"value": "Asimov, Isaac & Robert Silverberg", "count": 2, "score": 0.75} - ], - "total": 2 -} -``` - -## Response (200 OK) - -**Media items search:** -```json -[ - { - "id": "uuid", - "title": "Foundation", - "author": "Asimov, Isaac", - "library_id": "...", - "library_name": "E-Books" - } -] -``` - -**Field values search (autocomplete):** -```json -{ - "results": [ - {"value": "Asimov, Isaac", "count": 47, "score": 0.8} - ], - "total": 1 -} -``` - -## Error Responses - -| Code | Description | -| ---- | ---------------------------- | -| 400 | Invalid library_id | -| 400 | Missing library_id | -| 401 | Invalid or expired token | -| 404 | No results found | -``` - -#### 6.2 Update Bruno Collection - -**File:** `bruno/media-items/Search All Libraries.yml` - -**Update params section:** - -```yaml -params: - - name: q - value: "foundation" - type: query - disabled: false - - name: library_id - value: "{{library_id}}" - type: query - disabled: false - - name: author_filter - value: "" - type: query - disabled: true - - name: genre_filter - value: "" - type: query - disabled: true - - name: year_min - value: "" - type: query - disabled: true - - name: year_max - value: "" - type: query - disabled: true -``` - -**Add new Bruno files:** - -`bruno/media-items/Fuzzy Author Filter.yml` -`bruno/media-items/Fuzzy Genre Filter.yml` -`bruno/media-items/Combined Search and Filters.yml` -`bruno/media-items/Exact Match With Quotes.yml` -`bruno/media-items/Field Values Search - Authors.yml` -`bruno/media-items/scenarios/Unified Search Scenarios.yml` - -#### 6.3 Delete Deprecated Filtered Endpoint Documentation - -**Files to delete:** -- `docs/developer/api/media-items/filtered_media_items.md` (if exists) -- Remove `/api/media-items/filtered` from `docs/developer/api/api-reference.md` -- Remove `/api/media-items/filtered` from `docs/developer/api-reference.md` - ---- - -### Phase 8: No Changes Needed (Skip) - -**No changes required to:** -- `cmd/server/main.go` -- `cmd/server/tests/test_helpers_test.go` - -SearchService follows the pattern of `FiltersService` and `CollectionService` - it's created **inside** the handler constructor, not in main.go. This keeps handler dependencies self-contained. - -**Rationale:** -- Matches established pattern (`FiltersService`, `CollectionService`) -- Handler owns its service dependencies -- Simpler initialization - no handler-specific services in main.go -- More testable - ---- - -### Phase 9: Cleanup - -#### 7.1 Delete Deprecated Filtered Endpoint - -**File:** `internal/router/media.go` - -**Delete line 17:** -```go -protected.GET("/media-items/filtered", cfg.MediaHandler.ListMediaItemsFiltered) -``` - -**File:** `internal/handlers/media.go` - -**Delete handler `ListMediaItemsFiltered` (lines 705-766):** -```go -// DELETE THIS FUNCTION -func (mh *MediaHandler) ListMediaItemsFiltered(c *echo.Context) error { - ... -} -``` - -**File:** `internal/database/queries/queries.sql` - -**Delete query `ListMediaItemsFiltered` (lines 227-268):** -```sql --- DELETE THIS QUERY --- name: ListMediaItemsFiltered :many -... -``` - -**Regenerate Go code:** -```bash -go generate ./internal/database -``` - -#### 7.2 Delete Deprecated Tests - -**File:** `cmd/server/tests/filtering_test.go` - -**Delete entire file** (tests now covered by `search_unified_test.go`) - -**OR** update tests to use `/api/media-items/search` endpoint if some test cases are still valuable - -#### 7.3 Delete Deprecated Bruno Files - -**Delete:** `bruno/media-items/scenarios/Filter Media Items.yml` (if exists) - ---- - -### Phase 10: Verification - -#### 8.1 Compile Check - -```bash -go build ./... -cd web && npm run build -``` - -#### 8.2 Run Tests - -```bash -go test ./cmd/server/tests -v -run TestUnifiedSearch -go test ./cmd/server/tests -v # All tests should pass -``` - -#### 8.3 Manual Testing Checklist - -- [ ] Search box uses `/api/media-items/search` -- [ ] Fuzzy author filter works (asimov โ†’ Asimov, Isaac) -- [ ] Fuzzy genre filter works (scifi โ†’ Sci-Fi) -- [ ] Exact match with quotes works ("Foundation and Empire") -- [ ] Combined search + filters works -- [ ] Autocomplete dropdowns populate correctly -- [ ] Year range filter works (exact) -- [ ] Boolean filter works (exact) -- [ ] Saved filters load correctly -- [ ] Pagination works -- [ ] No errors in browser console -- [ ] No errors in server logs - -#### 8.4 Documentation Verification - -```bash -# Start server -podman compose up -d - -# Access docs at http://localhost:8080/docs -# Verify search endpoint documentation renders correctly -# Verify search finds new documentation -``` - ---- - -**Commit 3: Add search service** -git add internal/services/search.go -git commit -m "feat: add SearchService for unified search functionality - -- Create SearchService with SearchMediaItemsUnified method -- Add SearchFieldValues method for autocomplete dropdowns -- Add parseSearchQuery helper for quote detection -- Move all business logic from handler to service layer -- Follow established service pattern (FiltersService, CollectionService)" - -```bash -# Commit 1: Database schema (GIN indexes) -git add database/schema/schema.sql -git commit -m "feat: add GIN indexes for pg_trgm fuzzy search performance - -- Add gin_trgm_ops indexes on author, title, series, genre, language -- Improves fuzzy search performance on large libraries -- Required for unified search/filter endpoint" - -# Commit 2: SQL queries -git add internal/database/queries/queries.sql -git commit -m "feat: add unified search SQL query with fuzzy filters - -- Add SearchMediaItemsUnified query with all-fuzzy filters -- Add SearchFieldValues query for autocomplete dropdowns -- Support exact match with quotes detection -- Combine search + filters in single query" - -# Commit 3: Add search service -git add internal/services/search.go -git commit -m "feat: add SearchService for unified search functionality - -- Create SearchService with SearchMediaItemsUnified method -- Add SearchFieldValues method for autocomplete dropdowns -- Add parseSearchQuery helper for quote detection -- Move all business logic from handler to service layer -- Follow established service pattern (FiltersService, CollectionService)" - -# Commit 4: Regenerate database code -git add internal/database/queries.sql.go internal/database/models.go -git commit -m "chore: regenerate database code from queries.sql" - -# Commit 5: Update handler to use search service -git add internal/handlers/media.go -git commit -m "refactor: update SearchMediaItems handler to use SearchService - -- Add searchService to MediaHandler struct -- Create SearchService inside NewMediaHandler constructor -- Refactor SearchMediaItems to delegate to service layer -- Add handleUnifiedSearch method (thin wrapper) -- Add handleFieldValuesSearch method (thin wrapper) -- Handler now only extracts params and calls service -- Follows pattern of FiltersService and CollectionService" - -# Commit 6: Frontend templates -git add templates/bookshelf.templ -git commit -m "feat: fix search box to use /search endpoint with autocomplete - -- Change search box from /filtered to /search -- Change pagination buttons to use /search -- Add datalist elements for autocomplete -- Add Alpine.js event handlers for dropdown population" - -# Commit 8: Frontend TypeScript -git add web/src/bookshelf.ts -git commit -m "feat: add autocomplete dropdown support for filter fields - -- Add fetchFieldValues function for API calls -- Add fetchAuthorValues, fetchGenreValues, etc. -- Register functions globally for template access -- Populate datalist elements with fuzzy search results" - -# Commit 9: Tests -git add cmd/server/tests/search_unified_test.go -git commit -m "test: add comprehensive tests for unified search endpoint - -- Test fuzzy author/genre filters -- Test exact match with quotes -- Test combined search + filters -- Test field-specific search for dropdowns -- Test year range and boolean filters -- Use setupDeviceTest helper following PROJECT_GUIDELINES.md" - -# Commit 10: Documentation -git add docs/developer/api/media-items/search_media_items.md -git add bruno/media-items/Search\ All\ Libraries.yml -git add bruno/media-items/Fuzzy\ Author\ Filter.yml -git add bruno/media-items/Fuzzy\ Genre\ Filter.yml -git add bruno/media-items/Combined\ Search\ and\ Filters.yml -git add bruno/media-items/Exact\ Match\ With\ Quotes.yml -git add bruno/media-items/Field\ Values\ Search\ -\ Authors.yml -git commit -m "docs: update search API documentation with fuzzy filters - -- Document all-fuzzy filters (except years/booleans) -- Document exact match with quotes -- Document combined search + filters -- Document field-specific search for autocomplete -- Add comprehensive examples -- Update Bruno collection with new endpoints" - -# Commit 11: Delete deprecated code -git add internal/router/media.go -git add internal/handlers/media.go -git add internal/database/queries/queries.sql -git add internal/database/queries.sql.go -git add internal/database/models.go -git add cmd/server/tests/filtering_test.go -git commit -m "refactor: remove deprecated /filtered endpoint - -- Delete /media-items/filtered route registration -- Delete ListMediaItemsFiltered handler -- Delete ListMediaItemsFiltered SQL query -- Delete filtering_test.go (covered by search_unified_test.go) -- Regenerate database code after query deletion" - -# Commit 12: Final verification -git add . -git commit -m "chore: final verification of unified search implementation - -- All tests pass -- Documentation renders correctly -- Bruno collection updated -- No compilation errors -- Manual testing complete" -``` - ---- - -## Risk Mitigation - -### Potential Issues - -1. **Saved filters breaking:** Frontend-only, should work seamlessly -2. **Performance degradation:** GIN indexes should prevent this -3. **Breaking mobile apps:** `/filtered` endpoint will be deleted in Phase 9 -4. **Test coverage gaps:** Comprehensive tests in Phase 6 -5. **Handler integration incomplete:** Phase 4.1 handler updates were missing from original plan (NOW ADDED) - -### Rollback Plan - -If issues arise: -```bash -# Revert to previous commit -git revert HEAD - -# Or restore specific files -git show HEAD~1:internal/handlers/media.go > internal/handlers/media.go -git show HEAD~1:templates/bookshelf.templ > templates/bookshelf.templ -``` - ---- - -## Timeline Estimate - -- Phase 1 (Database): 30 minutes โœ… COMPLETED -- Phase 2 (SQL Queries): 1 hour โœ… COMPLETED -- Phase 3 (Search Service): 2 hours โœ… COMPLETED -- Phase 4 (Handler + Frontend): 2 hours โš ๏ธ PARTIAL (code specified, ready to implement) -- Phase 5 (Frontend Build): 10 minutes โŒ NOT STARTED -- Phase 6 (Tests): 2 hours โŒ NOT STARTED -- Phase 7 (Documentation): 1 hour โŒ NOT STARTED -- Phase 8 (No Changes): 0 minutes โœ… SKIPPED -- Phase 9 (Cleanup): 30 minutes โŒ NOT STARTED -- Phase 10 (Verification): 1 hour โŒ NOT STARTED - -**Total: ~10 hours** -**Completed: ~3.5 hours** (Phases 1-3) -**Remaining: ~3 hours** (Phases 4-10, all code specified, just implementation) - ---- - -## Success Criteria - -โœ… All business logic in `services/search.go` (service layer pattern) -โœ… SearchService created inside NewMediaHandler constructor (not main.go) -โœ… Handler is thin - only extracts params and calls service -โœ… All text filters use fuzzy matching (pg_trgm) -โœ… Exact match with quotes works -โœ… Combined search + filters work -โœ… Autocomplete dropdowns populate correctly -โœ… Years/booleans remain exact match -โœ… Saved filters load correctly -โœ… All tests pass using `setupDeviceTest` helper -โœ… Documentation updated -โœ… No breaking changes to saved filters -โœ… Deprecated `/filtered` endpoint removed -โœ… Bruno collection updated