bookhoard/UNIFIED_SEARCH_IMPLEMENTATION.md

# Unified Search and Filter Implementation Plan

## Executive Summary

**Goal:** Consolidate `/api/media-items/filtered` and `/api/media-items/search` endpoints into a single unified `/api/media-items/search` endpoint that supports:
- All-fuzzy filters (except years/booleans)
- Google-style "exact match in quotes" for search queries
- Field-specific fuzzy search for autocomplete dropdowns
- Combined search + filters functionality
- Backward compatibility with saved filters

**Approach:** Surgical, incremental changes that reuse existing code, following PROJECT_GUIDELINES.md strictly.

---

## Current State Analysis

### Existing Endpoints

#### 1. `/api/media-items/search` (internal/router/search.go:11)
- **Purpose:** Global fuzzy search across all libraries
- **Handler:** `SearchMediaItems` (internal/handlers/media.go:1419-1493)
- **Logic:**
  - Uses `SearchMediaItems` query (ILIKE pattern matching)
  - Falls back to `SearchMediaItemsFuzzy` query (pg_trgm `word_similarity()`)
  - Threshold: 0.3 similarity
- **Parameters:** `q`, `library_id`, `limit`, `offset`
- **Used by:** Search box (incorrectly - currently calls `/filtered`)

#### 2. `/api/media-items/filtered` (internal/router/media.go:17)
- **Purpose:** Exact match filtering
- **Handler:** `ListMediaItemsFiltered` (internal/handlers/media.go:705-766)
- **Logic:**
  - `ListMediaItemsFiltered` query (queries.sql:227-268)
  - Author/series: `ILIKE` (case-insensitive, NO wildcards)
  - Genre/language: `=` (exact match)
  - Years: `>=`, `<=` (exact range)
  - Boolean: exact match
- **Parameters:** `author_filter`, `series_filter`, `genre_filter`, `language_filter`, `year_min`, `year_max`, `has_cover`, `sort`, `limit`, `offset`
- **Used by:** All filter fields, search box (incorrectly)

### Current Frontend Implementation

**Search box** (templates/bookshelf.templ:73-83):
```html
<input name="search"
       hx-get="/api/media-items/filtered"  <!-- WRONG: Should be /search -->
       hx-trigger="keyup changed delay:300ms"
       hx-include="#filter-form">
```

**Filter fields** (templates/bookshelf.templ:90-117):
```html
<input name="author_filter"
       hx-get="/api/media-items/filtered"
       hx-include="#filter-form">

<input name="genre_filter"
       hx-get="/api/media-items/filtered"
       hx-include="#filter-form">
```

### Database Schema

**Indexes exist** (database/schema/schema.sql):
- B-tree indexes: author, genre, language, series, copyright_year (lines 365-373)
- GIN indexes: tags_search, contributors_search (lines 145-146)

**Missing:** GIN indexes for pg_trgm fuzzy search on text fields

### Saved Filters

**Implementation:** Frontend-only (web/src/bookshelf.ts:44-173)
- Stores filter config as JSON in database
- Loads filter config into form via `loadFilter()` method
- Makes API call to `/api/saved-filters/:id`
- Populates form fields with exact values

**Impact:** Should work seamlessly with new endpoint (only field names matter)

---

## Implementation Phases

### Phase 1: Database Schema - Add GIN Indexes

**File:** `database/schema/schema.sql`

**Add after line 146:**

```sql
-- Add GIN indexes for pg_trgm fuzzy search performance
CREATE INDEX IF NOT EXISTS idx_media_items_author_trgm
  ON media_items USING GIN (author gin_trgm_ops);
CREATE INDEX IF NOT EXISTS idx_media_items_title_trgm
  ON media_items USING GIN (title gin_trgm_ops);
CREATE INDEX IF NOT EXISTS idx_media_items_series_trgm
  ON media_items USING GIN (series gin_trgm_ops);
CREATE INDEX IF NOT EXISTS idx_media_items_genre_trgm
  ON media_items USING GIN (genre gin_trgm_ops);
CREATE INDEX IF NOT EXISTS idx_media_items_language_trgm
  ON media_items USING GIN (language gin_trgm_ops);
```

**Verification:**
```bash
podman compose down -v
podman compose up -d
podman exec bookhoard_db psql -U postgres -d bookhoard -c "\d+ media_items"
```

---

### Phase 2: SQL Queries - Add Unified Search Query

**File:** `internal/database/queries/queries.sql`

**Add after `SearchMediaItemsFuzzy` (line 460):**

```sql
-- name: SearchMediaItemsUnified :many
SELECT mi.*, l.name as library_name, lt.name as library_type_name
FROM media_items mi
JOIN libraries l ON mi.library_id = l.id
JOIN library_types lt ON l.library_type_id = lt.id
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
WHERE COALESCE(lv.is_visible, true) = true
  AND mi.library_id = sqlc.narg('library_id')
  -- Fuzzy author filter
  AND (sqlc.narg('author_filter') = '' OR word_similarity(sqlc.narg('author_filter'), COALESCE(mi.author, '')) > 0.3)
  -- Fuzzy series filter
  AND (sqlc.narg('series_filter') = '' OR word_similarity(sqlc.narg('series_filter'), COALESCE(mi.series, '')) > 0.3)
  -- Fuzzy genre filter
  AND (sqlc.narg('genre_filter') = '' OR word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')) > 0.3)
  -- Fuzzy language filter
  AND (sqlc.narg('language_filter') = '' OR word_similarity(sqlc.narg('language_filter'), COALESCE(mi.language, '')) > 0.3)
  -- Year range (exact)
  AND (sqlc.narg('year_min') = 0 OR mi.copyright_year >= sqlc.narg('year_min'))
  AND (sqlc.narg('year_max') = 0 OR mi.copyright_year <= sqlc.narg('year_max'))
  -- Boolean (exact)
  AND (sqlc.narg('has_cover') = false OR mi.cover_image_path IS NOT NULL)
  -- Search query (fuzzy or exact based on quotes)
  AND (
    sqlc.narg('search_query') = '' OR
    -- Fuzzy search (default)
    sqlc.narg('is_exact_search') = false AND (
      word_similarity(sqlc.narg('search_query'), mi.title) > 0.3 OR
      word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, '')) > 0.3 OR
      word_similarity(sqlc.narg('search_query'), COALESCE(mi.series, '')) > 0.3 OR
      EXISTS (
        SELECT 1 FROM unnest(mi.tags_search) AS tag
        WHERE word_similarity(sqlc.narg('search_query'), tag) > 0.3
        LIMIT 1
      ) OR
      EXISTS (
        SELECT 1 FROM unnest(mi.contributors_search) AS contributor
        WHERE word_similarity(sqlc.narg('search_query'), contributor) > 0.3
        LIMIT 1
      )
    ) OR
    -- Exact search (with quotes)
    sqlc.narg('is_exact_search') = true AND (
      mi.title ILIKE sqlc.narg('search_pattern') OR
      mi.author ILIKE sqlc.narg('search_pattern') OR
      mi.series ILIKE sqlc.narg('search_pattern') OR
      sqlc.narg('search_pattern') = ANY(mi.tags_search) OR
      sqlc.narg('search_pattern') = ANY(mi.contributors_search)
    )
  )
ORDER BY
  CASE
    WHEN sqlc.narg('search_query') != '' THEN
      GREATEST(
        CASE WHEN sqlc.narg('is_exact_search') = false THEN
          word_similarity(sqlc.narg('search_query'), mi.title)
        ELSE 0 END,
        CASE WHEN sqlc.narg('is_exact_search') = false THEN
          word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, ''))
        ELSE 0 END,
        word_similarity(sqlc.narg('author_filter'), COALESCE(mi.author, '')),
        word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, ''))
      )
    ELSE 0
  END DESC,
  mi.title ASC
LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset');
```

**Add field value search query (for autocomplete dropdowns):**

```sql
-- name: SearchFieldValues :many
SELECT DISTINCT
  CASE sqlc.narg('field_type')
    WHEN 'author' THEN mi.author
    WHEN 'genre' THEN mi.genre
    WHEN 'series' THEN mi.series
    WHEN 'language' THEN mi.language
  END as value,
  COUNT(*) as count,
  CASE sqlc.narg('field_type')
    WHEN 'author' THEN word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, ''))
    WHEN 'genre' THEN word_similarity(sqlc.narg('search_query'), COALESCE(mi.genre, ''))
    WHEN 'series' THEN word_similarity(sqlc.narg('search_query'), COALESCE(mi.series, ''))
    WHEN 'language' THEN word_similarity(sqlc.narg('search_query'), COALESCE(mi.language, ''))
  END as score
FROM media_items mi
WHERE mi.library_id = sqlc.narg('library_id')
  AND (
    (sqlc.narg('field_type') = 'author' AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, '')) > 0.3) OR
    (sqlc.narg('field_type') = 'genre' AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.genre, '')) > 0.3) OR
    (sqlc.narg('field_type') = 'series' AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.series, '')) > 0.3) OR
    (sqlc.narg('field_type') = 'language' AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.language, '')) > 0.3)
  )
GROUP BY value, score
HAVING value IS NOT NULL AND value != ''
ORDER BY score DESC, count DESC
LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset');
```

**Regenerate Go code:**
```bash
go generate ./internal/database
```

**Verify:** Check `internal/database/queries.sql.go` for new functions

---

### Phase 3: Create Search Service

**File:** `internal/services/search.go` (NEW)

```go
package services

import (
  "bookhoard/internal/database"
  "context"
  "strconv"
  "strings"

  "github.com/google/uuid"
  "github.com/jackc/pgx/v5/pgtype"
)

// SearchService handles all search and filter operations
type SearchService struct {
  db *database.Queries
}

// NewSearchService creates a new search service instance
func NewSearchService(db *database.Queries) *SearchService {
  return &SearchService{db: db}
}

// SearchParams contains parameters for unified search
type SearchParams struct {
  UserID         pgtype.UUID
  LibraryID      pgtype.UUID
  AuthorFilter   string
  SeriesFilter   string
  GenreFilter    string
  LanguageFilter string
  YearMin        int
  YearMax        int
  HasCover       bool
  SearchQuery    string
  Limit          int
  Offset         int
}

// parseSearchQuery detects quoted strings for exact match search
// Returns (isExact, processedQuery)
// Examples:
//   "asimov" → (false, "asimov")
//   "\"Asimov, Isaac\"" → (true, "Asimov, Isaac")
func (s *SearchService) parseSearchQuery(query string) (bool, string) {
  query = strings.TrimSpace(query)
  if strings.HasPrefix(query, "\"") && strings.HasSuffix(query, "\"") && len(query) >= 2 {
    return true, strings.Trim(query, "\"")
  }
  return false, query
}

// SearchMediaItemsUnified handles combined search + filters
// Supports:
// - Fuzzy text filters (author, series, genre, language)
// - Exact filters (year range, has_cover boolean)
// - Fuzzy search query (or exact match with quotes)
// - Combined search + filters
func (s *SearchService) SearchMediaItemsUnified(ctx context.Context, params SearchParams) ([]database.SearchMediaItemsUnifiedRow, error) {
  // Parse search query for exact match detection
  isExact, searchQuery := s.parseSearchQuery(params.SearchQuery)
  searchPattern := ""
  if isExact {
    searchPattern = "%" + searchQuery + "%"
  }

  // Build database parameters
  dbParams := database.SearchMediaItemsUnifiedParams{
    UserID:         params.UserID,
    LibraryID:      params.LibraryID,
    AuthorFilter:   pgtype.Text{String: params.AuthorFilter, Valid: true},
    SeriesFilter:   pgtype.Text{String: params.SeriesFilter, Valid: true},
    GenreFilter:    pgtype.Text{String: params.GenreFilter, Valid: true},
    LanguageFilter: pgtype.Text{String: params.LanguageFilter, Valid: true},
    YearMin:        pgtype.Int4{Int32: int32(params.YearMin), Valid: true},
    YearMax:        pgtype.Int4{Int32: int32(params.YearMax), Valid: true},
    HasCover:       pgtype.Bool{Bool: params.HasCover, Valid: true},
    SearchQuery:    pgtype.Text{String: searchQuery, Valid: true},
    IsExactSearch:  pgtype.Bool{Bool: isExact, Valid: true},
    SearchPattern:  pgtype.Text{String: searchPattern, Valid: isExact},
    Limit:          pgtype.Int4{Int32: int32(params.Limit), Valid: true},
    Offset:         pgtype.Int4{Int32: int32(params.Offset), Valid: true},
  }

  // Execute unified search query
  results, err := s.db.SearchMediaItemsUnified(ctx, dbParams)
  if err != nil {
    return nil, err
  }

  return results, nil
}

// FieldSearchParams contains parameters for field-specific search (autocomplete)
type FieldSearchParams struct {
  UserID      pgtype.UUID
  LibraryID   pgtype.UUID
  FieldType   string // "author", "genre", "series", "language"
  SearchQuery string
  Limit       int
  Offset      int
}

// FieldValue represents a single field value with metadata
type FieldValue struct {
  Value string
  Count int64
  Score float64
}

// SearchFieldValues handles field-specific search for autocomplete dropdowns
// Returns distinct values with counts and similarity scores
func (s *SearchService) SearchFieldValues(ctx context.Context, params FieldSearchParams) ([]FieldValue, error) {
  // Build database parameters
  dbParams := database.SearchFieldValuesParams{
    UserID:      params.UserID,
    LibraryID:   params.LibraryID,
    FieldType:   pgtype.Text{String: params.FieldType, Valid: true},
    SearchQuery: pgtype.Text{String: params.SearchQuery, Valid: true},
    Limit:       pgtype.Int4{Int32: params.Limit), Valid: true},
    Offset:      pgtype.Int4{Int32: params.Offset), Valid: true},
  }

  // Execute field values query
  results, err := s.db.SearchFieldValues(ctx, dbParams)
  if err != nil {
    return nil, err
  }

  // Convert to service-level response type
  fieldValues := make([]FieldValue, len(results))
  for i, r := range results {
    fieldValues[i] = FieldValue{
      Value: r.Value,
      Count: r.Count,
      Score: r.Score,
    }
  }

  return fieldValues, nil
}
```

**Verification:**
```bash
go build ./internal/handlers
```

**Note:** Router initialization will also need updating (see Phase 7)

#### 4.1 Fix Search Box Endpoint

**File:** `templates/bookshelf.templ`

**Change line 79:**
```html
<!-- BEFORE -->
<input ... hx-get="/api/media-items/filtered" ... >

<!-- AFTER -->
<input ... hx-get="/api/media-items/search" ... >
```

**Also fix pagination buttons (lines 298, 311):**
```html
<!-- BEFORE -->
hx-get="/api/media-items/filtered?library_id={ currentLibraryID }&limit={ limit }&offset={ offset - limit }"

<!-- AFTER -->
hx-get="/api/media-items/search?library_id={ currentLibraryID }&limit={ limit }&offset={ offset - limit }"
```

**Change filter field names (lines 92, 109, 126, etc.):**
```html
<!-- BEFORE -->
<input name="author_filter" ... hx-get="/api/media-items/filtered" ... >
<input name="genre_filter" ... hx-get="/api/media-items/filtered" ... >

<!-- AFTER -->
<input name="author_filter" ... hx-get="/api/media-items/search" ... >
<input name="genre_filter" ... hx-get="/api/media-items/search" ... >
```

#### 4.2 Add Autocomplete Dropdowns

**File:** `templates/bookshelf.templ`

**Replace author filter (lines 85-101):**

```html
<!-- Author Filter with Autocomplete -->
<div class="flex-1 min-w-[150px]">
  <label class="block text-sm font-medium mb-2" style="color: var(--text-secondary)">
    Author
  </label>
  <div class="relative" x-data="{ authorValues: [] }">
    <input
      type="text"
      name="author_filter"
      placeholder="Filter by author"
      class="w-full px-3 py-2 border rounded-lg"
      style="background-color: var(--bg-primary); color: var(--text-primary); border-color: var(--border);"
      hx-get="/api/media-items/search"
      hx-target="#books-grid"
      hx-trigger="change"
      hx-include="#filter-form"
      list="author-datalist"
      @focus="fetchAuthorValues($el)"
      @input.debounce.300ms="if($el.value === '') authorValues = []"
    />
    <datalist id="author-datalist">
      <template x-for="item in authorValues" :key="item.value">
        <option :value="item.value" x-text="`${item.value} (${item.count})`"></option>
      </template>
    </datalist>
  </div>
</div>
```

**Similar changes for genre, series, language filters**

#### 4.3 Add Frontend Functions

**File:** `web/src/bookshelf.ts`

**Add at end of file:**

```typescript
// Fetch field values for autocomplete dropdowns
async function fetchFieldValues(
  field: string,
  search: string,
  datalistId: string
): Promise<void> {
  const token = localStorage.getItem("token");
  if (!token) {
    console.error("Not authenticated");
    return;
  }

  if (search.length < 2) return; // Wait for at least 2 characters

  const currentLibraryId = (
    document.getElementById("library-select") as HTMLSelectElement
  )?.value;
  if (!currentLibraryId) {
    console.error("No library selected");
    return;
  }

  try {
    const response = await fetch(
      `/api/media-items/search?${field}=${encodeURIComponent(
        search
      )}&library_id=${currentLibraryId}&limit=50`,
      {
        headers: { Authorization: `Bearer ${token}` },
      }
    );

    if (!response.ok) {
      console.error("Failed to fetch field values");
      return;
    }

    const data = await response.json();

    // Update datalist
    const datalist = document.getElementById(datalistId);
    if (!datalist) {
      console.error(`Datalist ${datalistId} not found`);
      return;
    }

    // Clear existing options
    datalist.innerHTML = "";

    // Add new options
    data.results.forEach((item: { value: string; count: number }) => {
      const option = document.createElement("option");
      option.value = item.value;
      option.textContent = `${item.value} (${item.count})`;
      datalist.appendChild(option);
    });
  } catch (error) {
    console.error("Error fetching field values:", error);
  }
}

// Fetch author values for autocomplete
async function fetchAuthorValues(input: HTMLInputElement): Promise<void> {
  const search = input.value;
  await fetchFieldValues("authors", search, "author-datalist");
}

// Fetch genre values for autocomplete
async function fetchGenreValues(input: HTMLInputElement): Promise<void> {
  const search = input.value;
  await fetchFieldValues("genres", search, "genre-datalist");
}

// Fetch series values for autocomplete
async function fetchSeriesValues(input: HTMLInputElement): Promise<void> {
  const search = input.value;
  await fetchFieldValues("series", search, "series-datalist");
}

// Fetch language values for autocomplete
async function fetchLanguageValues(input: HTMLInputElement): Promise<void> {
  const search = input.value;
  await fetchFieldValues("languages", search, "language-datalist");
}

// Register functions globally
(window as any).fetchAuthorValues = fetchAuthorValues;
(window as any).fetchGenreValues = fetchGenreValues;
(window as any).fetchSeriesValues = fetchSeriesValues;
(window as any).fetchLanguageValues = fetchLanguageValues;
```

**Verify TypeScript compilation:**
```bash
cd web && npm run build
```

---

### Phase 5: Tests

**File:** `cmd/server/tests/search_unified_test.go` (NEW)

```go
package main

import (
  "bookhoard/internal/handlers"
  "encoding/json"
  "net/http"
  "net/http/httptest"
  "testing"

  "github.com/stretchr/testify/assert"
  "github.com/stretchr/testify/require"
)

func TestUnifiedSearch(t *testing.T) {
  setup := setupDeviceTest(t)
  defer setup.Server.Close()

  libraryID := setup.CreateLibrary(t, "Test Search Library", "ebooks")
  _ = setup.CreateDevice(t, "Test Search Device", "koreader", "search-test-123")

  t.Run("Fuzzy author filter", func(t *testing.T) {
    req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&author_filter=asimov", nil)
    req.Header.Set("Authorization", "Bearer "+setup.UserToken)
    rec := httptest.NewRecorder()
    setup.Server.Config.Handler.ServeHTTP(rec, req)

    assert.Equal(t, http.StatusOK, rec.Code, "Should fuzzy match author")
  })

  t.Run("Fuzzy genre filter", func(t *testing.T) {
    req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&genre_filter=scifi", nil)
    req.Header.Set("Authorization", "Bearer "+setup.UserToken)
    rec := httptest.NewRecorder()
    setup.Server.Config.Handler.ServeHTTP(rec, req)

    assert.Equal(t, http.StatusOK, rec.Code, "Should fuzzy match genre")
  })

  t.Run("Exact match with quotes", func(t *testing.T) {
    req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&q=%22Foundation%20and%20Empire%22", nil)
    req.Header.Set("Authorization", "Bearer "+setup.UserToken)
    rec := httptest.NewRecorder()
    setup.Server.Config.Handler.ServeHTTP(rec, req)

    assert.Equal(t, http.StatusOK, rec.Code, "Should exact match quoted query")
  })

  t.Run("Combined search + filters", func(t *testing.T) {
    req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&q=foundation&author_filter=asimov", nil)
    req.Header.Set("Authorization", "Bearer "+setup.UserToken)
    rec := httptest.NewRecorder()
    setup.Server.Config.Handler.ServeHTTP(rec, req)

    assert.Equal(t, http.StatusOK, rec.Code, "Should combine search and filters")
  })

  t.Run("Field-specific search for dropdown - authors", func(t *testing.T) {
    req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&authors=asimov", nil)
    req.Header.Set("Authorization", "Bearer "+setup.UserToken)
    rec := httptest.NewRecorder()
    setup.Server.Config.Handler.ServeHTTP(rec, req)

    assert.Equal(t, http.StatusOK, rec.Code, "Should return author values")

    var response struct {
      Results []struct {
        Value string  `json:"value"`
        Count int64   `json:"count"`
        Score float64 `json:"score"`
      } `json:"results"`
      Total int `json:"total"`
    }
    err := json.Unmarshal(rec.Body.Bytes(), &response)
    require.NoError(t, err, "Should unmarshal field values response")
    assert.Greater(t, len(response.Results), 0, "Should have results")
  })

  t.Run("Year range filter (exact)", func(t *testing.T) {
    req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&year_min=2000&year_max=2020", nil)
    req.Header.Set("Authorization", "Bearer "+setup.UserToken)
    rec := httptest.NewRecorder()
    setup.Server.Config.Handler.ServeHTTP(rec, req)

    assert.Equal(t, http.StatusOK, rec.Code, "Should filter by year range")
  })

  t.Run("Boolean filter (exact)", func(t *testing.T) {
    req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&has_cover=true", nil)
    req.Header.Set("Authorization", "Bearer "+setup.UserToken)
    rec := httptest.NewRecorder()
    setup.Server.Config.Handler.ServeHTTP(rec, req)

    assert.Equal(t, http.StatusOK, rec.Code, "Should filter by has_cover")
  })

  t.Run("Missing library_id", func(t *testing.T) {
    req := httptest.NewRequest("GET", "/api/media-items/search?q=test", nil)
    req.Header.Set("Authorization", "Bearer "+setup.UserToken)
    rec := httptest.NewRecorder()
    setup.Server.Config.Handler.ServeHTTP(rec, req)

    assert.Equal(t, http.StatusBadRequest, rec.Code, "Should require library_id")
  })
}
```

**Verify tests:**
```bash
go test ./cmd/server/tests -v -run TestUnifiedSearch
```

---

### Phase 6: Documentation Updates

#### 6.1 Update API Documentation

**File:** `docs/developer/api/media-items/search_media_items.md`

**Replace entire content:**

```markdown
# Search Media Items (Unified)

Search and filter media items with fuzzy matching support.

**Note:** All text filters use fuzzy matching via PostgreSQL pg_trgm (threshold: 0.3 similarity). This handles typos and partial matches automatically. Use quotes for exact match.

**Endpoint**: `GET /api/media-items/search`
**Auth**: Required

## Query Parameters

### Search Parameters

| Parameter  | Type    | Required | Description                                          |
| ---------- | ------- | -------- | ---------------------------------------------------- |
| q          | string  | No       | Search query (fuzzy by default, exact in quotes)     |
| library_id | string  | Yes      | Filter to specific library (UUID)                    |
| limit      | integer | No       | Number of results (default 50, max 200)              |
| offset     | integer | No       | Number to skip for pagination                        |

### Filter Parameters (All Fuzzy Except Years/Booleans)

| Parameter      | Type    | Description                                         |
| -------------- | ------- | --------------------------------------------------- |
| author_filter  | string  | Fuzzy match author field                            |
| series_filter  | string  | Fuzzy match series field                            |
| genre_filter   | string  | Fuzzy match genre field                             |
| language_filter| string  | Fuzzy match language field                          |
| year_min       | integer | Minimum copyright year (exact range)                |
| year_max       | integer | Maximum copyright year (exact range)                |
| has_cover      | boolean | Filter by cover image presence (exact boolean)      |

### Autocomplete Parameters (Field-Specific Search)

| Parameter  | Type   | Description                                    |
| ---------- | ------ | ---------------------------------------------- |
| authors    | string | Search author values for autocomplete dropdown |
| genres     | string | Search genre values for autocomplete dropdown  |
| series     | string | Search series values for autocomplete dropdown |
| languages  | string | Search language values for autocomplete        |

## Request Headers

| Header        | Type   | Required | Description  |
| ------------- | ------ | -------- | ------------ |
| Authorization | string | Yes      | Bearer token |

## Search Behavior

### Fuzzy Search (Default)

Handles typos and partial matches automatically:

- `"asimov"` → matches "Asimov, Isaac", "Asimov, Foundation"
- `"scifi"` → matches "Sci-Fi", "Science Fiction"
- `"azimov"` → matches "Asimov, Isaac" (typo tolerance)

### Exact Search (With Quotes)

Use double quotes for exact phrase matching:

- `"\"Foundation and Empire\""` → only "Foundation and Empire"
- `"\"Asimov, Isaac\""` → only "Asimov, Isaac"

### Filter Behavior

**Text filters (fuzzy):**
- `author_filter=asimov` → fuzzy matches author field
- `genre_filter=scifi` → fuzzy matches genre field

**Exact filters:**
- `year_min=2000&year_max=2010` → exact year range
- `has_cover=true` → exact boolean match

## Example Requests

### 1. Global Fuzzy Search

Search all fields for "foundation":

```http
GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&q=foundation
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
```

### 2. Fuzzy Author Filter

Find books by "asimov" (matches "Asimov, Isaac"):

```http
GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&author_filter=asimov
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
```

### 3. Combined Search + Filters

Search "foundation" within books by "asimov":

```http
GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&q=foundation&author_filter=asimov
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
```

### 4. Exact Match with Quotes

Exact phrase search:

```http
GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&q="Foundation%20and%20Empire"
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
```

### 5. Multiple Fuzzy Filters

Fiction books from 2000-2010:

```http
GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&genre_filter=fiction&year_min=2000&year_max=2010
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
```

### 6. Field-Specific Search (Autocomplete)

Get author values for dropdown:

```http
GET /api/media-items/search?library_id=123e4567-e89b-12d3-a456-426614174000&authors=asimov&limit=50
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
```

**Response:**
```json
{
  "results": [
    {"value": "Asimov, Isaac", "count": 47, "score": 0.8},
    {"value": "Asimov, Isaac & Robert Silverberg", "count": 2, "score": 0.75}
  ],
  "total": 2
}
```

## Response (200 OK)

**Media items search:**
```json
[
  {
    "id": "uuid",
    "title": "Foundation",
    "author": "Asimov, Isaac",
    "library_id": "...",
    "library_name": "E-Books"
  }
]
```

**Field values search (autocomplete):**
```json
{
  "results": [
    {"value": "Asimov, Isaac", "count": 47, "score": 0.8}
  ],
  "total": 1
}
```

## Error Responses

| Code | Description                  |
| ---- | ---------------------------- |
| 400  | Invalid library_id           |
| 400  | Missing library_id           |
| 401  | Invalid or expired token     |
| 404  | No results found             |
```

#### 6.2 Update Bruno Collection

**File:** `bruno/media-items/Search All Libraries.yml`

**Update params section:**

```yaml
params:
  - name: q
    value: "foundation"
    type: query
    disabled: false
  - name: library_id
    value: "{{library_id}}"
    type: query
    disabled: false
  - name: author_filter
    value: ""
    type: query
    disabled: true
  - name: genre_filter
    value: ""
    type: query
    disabled: true
  - name: year_min
    value: ""
    type: query
    disabled: true
  - name: year_max
    value: ""
    type: query
    disabled: true
```

**Add new Bruno files:**

`bruno/media-items/Fuzzy Author Filter.yml`
`bruno/media-items/Fuzzy Genre Filter.yml`
`bruno/media-items/Combined Search and Filters.yml`
`bruno/media-items/Exact Match With Quotes.yml`
`bruno/media-items/Field Values Search - Authors.yml`
`bruno/media-items/scenarios/Unified Search Scenarios.yml`

#### 6.3 Delete Deprecated Filtered Endpoint Documentation

**Files to delete:**
- `docs/developer/api/media-items/filtered_media_items.md` (if exists)
- Remove `/api/media-items/filtered` from `docs/developer/api/api-reference.md`
- Remove `/api/media-items/filtered` from `docs/developer/api-reference.md`

---

### Phase 7: Update Handler Initialization

**Files to update:**
- `cmd/server/main.go` (line ~122-126)
- `cmd/server/tests/test_helpers_test.go` (line ~468-472)

**In both files, find the MediaHandler initialization:**

```go
// BEFORE
mediaHandler := handlers.NewMediaHandler(queries, libraryService, worker)

// AFTER
searchService := services.NewSearchService(queries)
mediaHandler := handlers.NewMediaHandler(queries, libraryService, searchService, worker)
```

**Note:** This follows the same pattern as `conversionService` which is created in main.go (line 118) and passed to handlers.

**Verify compilation:**
```bash
go build ./cmd/server
go test ./cmd/server/tests -run TestNonExistent # Compile test only
```

---

### Phase 8: Cleanup

#### 7.1 Delete Deprecated Filtered Endpoint

**File:** `internal/router/media.go`

**Delete line 17:**
```go
protected.GET("/media-items/filtered", cfg.MediaHandler.ListMediaItemsFiltered)
```

**File:** `internal/handlers/media.go`

**Delete handler `ListMediaItemsFiltered` (lines 705-766):**
```go
// DELETE THIS FUNCTION
func (mh *MediaHandler) ListMediaItemsFiltered(c *echo.Context) error {
  ...
}
```

**File:** `internal/database/queries/queries.sql`

**Delete query `ListMediaItemsFiltered` (lines 227-268):**
```sql
-- DELETE THIS QUERY
-- name: ListMediaItemsFiltered :many
...
```

**Regenerate Go code:**
```bash
go generate ./internal/database
```

#### 7.2 Delete Deprecated Tests

**File:** `cmd/server/tests/filtering_test.go`

**Delete entire file** (tests now covered by `search_unified_test.go`)

**OR** update tests to use `/api/media-items/search` endpoint if some test cases are still valuable

#### 7.3 Delete Deprecated Bruno Files

**Delete:** `bruno/media-items/scenarios/Filter Media Items.yml` (if exists)

---

### Phase 10: Verification

#### 8.1 Compile Check

```bash
go build ./...
cd web && npm run build
```

#### 8.2 Run Tests

```bash
go test ./cmd/server/tests -v -run TestUnifiedSearch
go test ./cmd/server/tests -v  # All tests should pass
```

#### 8.3 Manual Testing Checklist

- [ ] Search box uses `/api/media-items/search`
- [ ] Fuzzy author filter works (asimov → Asimov, Isaac)
- [ ] Fuzzy genre filter works (scifi → Sci-Fi)
- [ ] Exact match with quotes works ("Foundation and Empire")
- [ ] Combined search + filters works
- [ ] Autocomplete dropdowns populate correctly
- [ ] Year range filter works (exact)
- [ ] Boolean filter works (exact)
- [ ] Saved filters load correctly
- [ ] Pagination works
- [ ] No errors in browser console
- [ ] No errors in server logs

#### 8.4 Documentation Verification

```bash
# Start server
podman compose up -d

# Access docs at http://localhost:8080/docs
# Verify search endpoint documentation renders correctly
# Verify search finds new documentation
```

---

**Commit 3: Add search service**
git add internal/services/search.go
git commit -m "feat: add SearchService for unified search functionality

- Create SearchService with SearchMediaItemsUnified method
- Add SearchFieldValues method for autocomplete dropdowns
- Add parseSearchQuery helper for quote detection
- Move all business logic from handler to service layer
- Follow established service pattern (FiltersService, CollectionService)"

```bash
# Commit 1: Database schema (GIN indexes)
git add database/schema/schema.sql
git commit -m "feat: add GIN indexes for pg_trgm fuzzy search performance

- Add gin_trgm_ops indexes on author, title, series, genre, language
- Improves fuzzy search performance on large libraries
- Required for unified search/filter endpoint"

# Commit 2: SQL queries
git add internal/database/queries/queries.sql
git commit -m "feat: add unified search SQL query with fuzzy filters

- Add SearchMediaItemsUnified query with all-fuzzy filters
- Add SearchFieldValues query for autocomplete dropdowns
- Support exact match with quotes detection
- Combine search + filters in single query"

# Commit 3: Add search service
git add internal/services/search.go
git commit -m "feat: add SearchService for unified search functionality

- Create SearchService with SearchMediaItemsUnified method
- Add SearchFieldValues method for autocomplete dropdowns
- Add parseSearchQuery helper for quote detection
- Move all business logic from handler to service layer
- Follow established service pattern (FiltersService, CollectionService)"

# Commit 4: Regenerate database code
git add internal/database/queries.sql.go internal/database/models.go
git commit -m "chore: regenerate database code from queries.sql"

# Commit 5: Update handler to use search service
git add internal/handlers/media.go
git commit -m "refactor: update SearchMediaItems handler to use SearchService

- Add searchService to MediaHandler struct
- Update NewMediaHandler constructor
- Refactor SearchMediaItems to delegate to service layer
- Add handleUnifiedSearch method (thin wrapper)
- Add handleFieldValuesSearch method (thin wrapper)
- Handler now only extracts params and calls service"

# Commit 6: Update handler initialization
git add cmd/server/main.go cmd/server/tests/test_helpers_test.go
git commit -m "refactor: add SearchService to handler initialization

- Instantiate SearchService in main.go and test_helpers_test.go
- Pass searchService to MediaHandler constructor
- Follow existing pattern (like conversionService)
- Update both production and test initialization"

# Commit 7: Frontend templates
git add templates/bookshelf.templ
git commit -m "feat: fix search box to use /search endpoint with autocomplete

- Change search box from /filtered to /search
- Change pagination buttons to use /search
- Add datalist elements for autocomplete
- Add Alpine.js event handlers for dropdown population"

# Commit 8: Frontend TypeScript
git add web/src/bookshelf.ts
git commit -m "feat: add autocomplete dropdown support for filter fields

- Add fetchFieldValues function for API calls
- Add fetchAuthorValues, fetchGenreValues, etc.
- Register functions globally for template access
- Populate datalist elements with fuzzy search results"

# Commit 9: Tests
git add cmd/server/tests/search_unified_test.go
git commit -m "test: add comprehensive tests for unified search endpoint

- Test fuzzy author/genre filters
- Test exact match with quotes
- Test combined search + filters
- Test field-specific search for dropdowns
- Test year range and boolean filters
- Use setupTestServer helper following PROJECT_GUIDELINES.md"

# Commit 10: Documentation
git add docs/developer/api/media-items/search_media_items.md
git add bruno/media-items/Search\ All\ Libraries.yml
git add bruno/media-items/Fuzzy\ Author\ Filter.yml
git add bruno/media-items/Fuzzy\ Genre\ Filter.yml
git add bruno/media-items/Combined\ Search\ and\ Filters.yml
git add bruno/media-items/Exact\ Match\ With\ Quotes.yml
git add bruno/media-items/Field\ Values\ Search\ -\ Authors.yml
git commit -m "docs: update search API documentation with fuzzy filters

- Document all-fuzzy filters (except years/booleans)
- Document exact match with quotes
- Document combined search + filters
- Document field-specific search for autocomplete
- Add comprehensive examples
- Update Bruno collection with new endpoints"

# Commit 11: Delete deprecated code
git add internal/router/media.go
git add internal/handlers/media.go
git add internal/database/queries/queries.sql
git add internal/database/queries.sql.go
git add internal/database/models.go
git add cmd/server/tests/filtering_test.go
git commit -m "refactor: remove deprecated /filtered endpoint

- Delete /media-items/filtered route registration
- Delete ListMediaItemsFiltered handler
- Delete ListMediaItemsFiltered SQL query
- Delete filtering_test.go (covered by search_unified_test.go)
- Regenerate database code after query deletion"

# Commit 12: Final verification
git add .
git commit -m "chore: final verification of unified search implementation

- All tests pass
- Documentation renders correctly
- Bruno collection updated
- No compilation errors
- Manual testing complete"
```

---

## Risk Mitigation

### Potential Issues

1. **Saved filters breaking:** Frontend-only, should work seamlessly
2. **Performance degradation:** GIN indexes should prevent this
3. **Breaking mobile apps:** `/filtered` endpoint will be deleted
4. **Test coverage gaps:** Comprehensive tests in Phase 5

### Rollback Plan

If issues arise:
```bash
# Revert to previous commit
git revert HEAD

# Or restore specific files
git show HEAD~1:internal/handlers/media.go > internal/handlers/media.go
git show HEAD~1:templates/bookshelf.templ > templates/bookshelf.templ
```

---

## Timeline Estimate

- Phase 1 (Database): 30 minutes
- Phase 2 (SQL Queries): 1 hour
- Phase 3 (Search Service): 2 hours
- Phase 4 (Handler Updates): 1 hour
- Phase 5 (Router Updates): 15 minutes
- Phase 6 (Frontend): 2 hours
- Phase 7 (TypeScript): 1 hour
- Phase 8 (Tests): 2 hours
- Phase 9 (Documentation): 1 hour
- Phase 10 (Cleanup): 30 minutes
- Phase 11 (Verification): 1 hour

**Total: ~12 hours**

---

## Success Criteria

✅ All business logic in `services/search.go` (service layer pattern)
✅ Handler is thin - only extracts params and calls service
✅ All text filters use fuzzy matching (pg_trgm)
✅ Exact match with quotes works
✅ Combined search + filters work
✅ Autocomplete dropdowns populate correctly
✅ Years/booleans remain exact match
✅ Saved filters load correctly
✅ All tests pass using `setupTestServer` helper
✅ Documentation updated
✅ No breaking changes to saved filters
✅ Deprecated `/filtered` endpoint removed
✅ Bruno collection updated