# Implementation Plan: Replace Genre Filter with Tags Filter

## Status: 📋 Planning Phase
**Created:** 2026-03-25
**Type:** Full-Stack Task
**Priority:** Medium
**Estimated Time:** 2-3 hours

---

## 🎯 Overview

Replace the `genre_filter` with `tags_filter` throughout the application. The `genre` field in the database is currently NULL for all Calibre imports because Calibre uses tags for genre classification. By filtering tags instead, we leverage existing data without requiring a schema migration.

**Key Decision:** Keep `genre_filter` in the API for backward compatibility but remove it from the frontend UI.

---

## 📊 Current State Analysis

### Database State
```sql
-- Current data (from investigation):
SELECT COUNT(*) as total, COUNT(genre) as with_genre, COUNT(tags) as with_tags FROM media_items;
-- Result: 17 total books, 0 with genre, 17 with tags
```

### Problem
- **Genre field:** NULL for all imported books (Calibre doesn't populate it)
- **Tags field:** Populated with Calibre tags (including genre-like tags)
- **Filter behavior:** genre_filter returns 0 results (no matches on NULL)
- **User impact:** Cannot filter books by genre/category

### Root Cause
Calibre treats tags as genre classification. Example tags:
```
["Science Fiction", "Adventure", "Dystopian", "Favorites"]
```

The current genre filter expects a single `genre` text field, which is always NULL.

---

## 🔄 Proposed Solution

### Option B: Filter Tags Instead (Selected)

**Why this approach:**
- ✅ No database migration required (zero downtime)
- ✅ Works immediately with existing Calibre data
- ✅ Low risk (isolated changes)
- ✅ Calibre users already understand this model
- ✅ Faster to implement

**Trade-offs:**
- ⚠️ Tags may contain non-genre values (e.g., "Favorites", "To Read")
- ⚠️ User filters by "Tags" instead of "Genre" in UI

**Why acceptable:**
- Calibre has the same "con" and it's not a problem there
- Users can filter their own tags as well (feature, not bug)
- API backward compatibility maintained (genre_filter still works)

---

## 📋 Implementation Checklist

### Phase 1: Backend - SQL Layer (15 minutes)
- [ ] Add `tags_filter` parameter to `SearchMediaItemsUnified` query
- [ ] Update filter logic from `word_similarity()` to `= ANY()` for array matching
- [ ] Keep `genre_filter` for backward compatibility
- [ ] Regenerate Go code with `sqlc generate`

### Phase 2: Backend - Service Layer (20 minutes)
- [ ] Add `TagsFilter string` to `SearchParams` struct
- [ ] Update `dbParams` building to include tags_filter
- [ ] Update handler to extract `tags_filter` query parameter
- [ ] Keep existing `genre_filter` extraction (backward compat)

### Phase 3: Backend - Tags Autocomplete API (30 minutes)
- [ ] Add `SearchTagsValues` SQL query
- [ ] Add "tags" case to `SearchFieldValues` service
- [ ] Add tags route to handler (autocomplete support)
- [ ] Test tags autocomplete endpoint

### Phase 4: Frontend - TypeScript (20 minutes)
- [ ] Rename `fetchGenreValues()` → `fetchTagValues()` in bookshelf.ts
- [ ] Update field id from "genre" → "tags" in custom-section-builder.ts
- [ ] Update collection rules from "genre" → "tags" in collection-rules.ts
- [ ] No other TS files need changes (search.ts is header search only)

### Phase 5: Frontend - Templates (15 minutes)
- [ ] Update bookshelf template (genre → tags labels/inputs)
- [ ] Change datalist id from "genre-datalist" → "tags-datalist"
- [ ] Update @input handler from `fetchGenreValues` → `fetchTagValues`
- [ ] Regenerate template Go files with `templ generate`

### Phase 6: Bruno Collection Updates (15 minutes)
- [ ] Create `Fuzzy Tags Filter.yml` (rename/update genre filter test)
- [ ] Create `Field Values Search - Tags.yml` (rename genre autocomplete)
- [ ] Update `Combined Search and Filters.yml` (genre → tags_filter)
- [ ] Add documentation for tags filter behavior

### Phase 7: Documentation (20 minutes)
- [ ] Create `docs/developer/api/media-items/search_media_items.md` (or update existing)
- [ ] Document tags_filter parameter
- [ ] Document backward compatibility with genre_filter
- [ ] Add examples of tags filtering
- [ ] Update any relevant user documentation

### Phase 8: Integration Tests (30 minutes)
- [ ] Create `tags_filter_test.go` in `cmd/server/tests/`
- [ ] Test with `setupTestServer()` helper from `test_helpers_test.go`
- [ ] Test tags filter returns matching books
- [ ] Test tags autocomplete endpoint
- [ ] Test backward compatibility (genre_filter still works)
- [ ] Test combined filters (tags + author + series)

### Phase 9: Verification & Testing (20 minutes)
- [ ] Run full test suite: `go test ./... -v`
- [ ] Test tags filter in browser (manual QA)
- [ ] Verify tags autocomplete works
- [ ] Verify backward compatibility (genre_filter)
- [ ] Check no regressions in existing filters

---

## 🔧 Technical Changes

### 1. SQL Query Changes

**File:** `internal/database/queries/queries.sql`

**Location:** Line 435-437 (SearchMediaItemsUnified query)

**Change:**
```sql
-- BEFORE:
-- Fuzzy genre filter
AND (sqlc.narg('genre_filter') = '' OR word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')) > 0.3)

-- AFTER:
-- Fuzzy genre filter (kept for backward compatibility)
AND (sqlc.narg('genre_filter') = '' OR word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')) > 0.3)
-- Tags filter (NEW - matches against tags array)
AND (sqlc.narg('tags_filter') = '' OR sqlc.narg('tags_filter') = ANY(mi.tags_search))
```

**New Query:** Add `SearchTagsValues` for autocomplete
```sql
-- name: SearchTagsValues :many
SELECT
  unnest(mi.tags_search) as value,
  COUNT(*) as count,
  word_similarity(sqlc.narg('search_query'), unnest(mi.tags_search))::float8 as score
FROM media_items mi
JOIN libraries l ON mi.library_id = l.id
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
WHERE COALESCE(lv.is_visible, true) = true
  AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
  AND unnest(mi.tags_search) IS NOT NULL
  AND word_similarity(sqlc.narg('search_query'), unnest(mi.tags_search)) > 0.3
GROUP BY unnest(mi.tags_search), word_similarity(sqlc.narg('search_query'), unnest(mi.tags_search))::float8
ORDER BY word_similarity(sqlc.narg('search_query'), unnest(mi.tags_search))::float8 DESC, COUNT DESC
LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset');
```

### 2. Service Layer Changes

**File:** `internal/services/search.go`

**Change 1:** Update SearchParams struct (line 22-36)
```go
type SearchParams struct {
    UserID         pgtype.UUID
    LibraryID      pgtype.UUID
    AuthorFilter   string
    SeriesFilter   string
    GenreFilter    string  // KEPT for backward compatibility
    TagsFilter     string  // NEW
    LanguageFilter string
    YearMin        int
    YearMax        int
    HasCover       bool
    SearchQuery    string
    Sort           string
    Limit          int
    Offset         int
}
```

**Change 2:** Update dbParams building (line 67-82)
```go
dbParams := database.SearchMediaItemsUnifiedParams{
    UserID:         params.UserID,
    LibraryID:      params.LibraryID,
    AuthorFilter:   pgtype.Text{String: params.AuthorFilter, Valid: params.AuthorFilter != ""},
    SeriesFilter:   pgtype.Text{String: params.SeriesFilter, Valid: params.SeriesFilter != ""},
    GenreFilter:    pgtype.Text{String: params.GenreFilter, Valid: params.GenreFilter != ""},  // KEPT
    TagsFilter:     pgtype.Text{String: params.TagsFilter, Valid: params.TagsFilter != ""},   // NEW
    LanguageFilter: pgtype.Text{String: params.LanguageFilter, Valid: params.LanguageFilter != ""},
    // ... rest of params
}
```

**Change 3:** Add tags case to SearchFieldValues (after line 151)
```go
case "tags":
    results, err := s.db.SearchTagsValues(ctx, database.SearchTagsValuesParams{
        SearchQuery: pgtype.Text{String: params.SearchQuery, Valid: true},
        UserID:      params.UserID,
        LibraryID:   params.LibraryID,
        Limit:       pgtype.Int4{Int32: int32(params.Limit), Valid: true},
        Offset:      pgtype.Int4{Int32: int32(params.Offset), Valid: true},
    })
    if err != nil {
        return nil, err
    }
    fieldValues := make([]FieldValue, len(results))
    for i, r := range results {
        fieldValues[i] = FieldValue{Value: r.Value.String, Count: r.Count, Score: r.Score}
    }
    return fieldValues, nil
```

### 3. Handler Changes

**File:** `internal/handlers/media.go`

**Change 1:** Extract tags_filter parameter (after line 1414)
```go
// Extract filter parameters
authorFilter := c.QueryParam("author_filter")
seriesFilter := c.QueryParam("series_filter")
genreFilter := c.QueryParam("genre_filter")      // KEPT for backward compatibility
tagsFilter := c.QueryParam("tags_filter")        // NEW
languageFilter := c.QueryParam("language_filter")
```

**Change 2:** Add to SearchParams (line 1426-1440)
```go
params := services.SearchParams{
    UserID:         userID.ID,
    LibraryID:      libUUID,
    SearchQuery:    query,
    AuthorFilter:   authorFilter,
    SeriesFilter:   seriesFilter,
    GenreFilter:    genreFilter,   // KEPT
    TagsFilter:     tagsFilter,    // NEW
    LanguageFilter: languageFilter,
    // ... rest of params
}
```

**Change 3:** Add tags autocomplete route (after line 1379)
```go
if tags := c.QueryParam("tags"); tags != "" {
    return mh.handleFieldValuesSearch(c, userID.ID, "tags", tags)
}
```

### 4. Frontend TypeScript Changes

**File:** `web/src/bookshelf.ts`

**Change:** Line 297-300
```typescript
// AFTER (add new function, keep old commented out):
async fetchTagValues(input: HTMLInputElement): Promise<void> {
  await this.fetchFieldValues("tags", input.value, "tags-datalist");
}

// TODO: Uncomment if genre field is populated in future
// async fetchGenreValues(input: HTMLInputElement): Promise<void> {
//   await this.fetchFieldValues("genre", input.value, "genre-datalist");
// }
```

**File:** `web/src/custom-section-builder.ts`

**Change:** Line 49-50
```typescript
// AFTER (add new field, keep old commented out):
[
  { id: "tags", label: "Tags" },
  // TODO: Uncomment if genre field is populated in future
  // { id: "genre", label: "Genre" },
  // ... other fields
]
```

**File:** `web/src/collection-rules.ts`

**Change:** Line 45
```typescript
// AFTER (add new field, keep old commented out):
const ruleFields = {
  tags: "Tags",
  // TODO: Uncomment if genre field is populated in future
  // genre: "Genre",
  // ... other fields
};
```

### 5. Template Changes

**File:** `templates/bookshelf.templ` (source file)

**Locate:** Find the genre filter input section (around line 131 in generated Go file)

**Change:**
```html
<!-- AFTER: Add tags filter, comment out genre filter for future use -->
<!-- Tags Filter with Autocomplete -->
<div class="flex-1 min-w-[150px]">
  <label class="block text-sm font-medium mb-2" style="color: var(--text-secondary)">Tags</label>
  <input type="text" name="tags_filter" placeholder="Filter by tags"
    class="w-full px-3 py-2 border rounded-lg"
    style="background-color: var(--bg-primary); color: var(--text-primary); border-color: var(--border);"
    hx-get="/api/media-items/search" hx-target="#books-grid"
    hx-trigger="keyup[key=='Enter'] from:#filter-form" hx-include="#filter-form"
    list="tags-datalist" @input.debounce.300ms="if($el.value.length >= 2) fetchTagValues($el)">
  <datalist id="tags-datalist"></datalist>
</div>

<!-- TODO: Uncomment if genre field is populated in future -->
{{- /* Genre Filter with Autocomplete */}
{{- /* <div class="flex-1 min-w-[150px]"> */}
{{- /*   <label class="block text-sm font-medium mb-2" style="color: var(--text-secondary)">Genre</label> */}
{{- /*   <input type="text" name="genre_filter" placeholder="Filter by genre" */}}
{{- /*     class="w-full px-3 py-2 border rounded-lg" */}}
{{- /*     style="background-color: var(--bg-primary); color: var(--text-primary); border-color: var(--border);" */}}
{{- /*     hx-get="/api/media-items/search" hx-target="#books-grid" */}}
{{- /*     hx-trigger="keyup[key=='Enter'] from:#filter-form" hx-include="#filter-form" */}}
{{- /*     list="genre-datalist" @input.debounce.300ms="if($el.value.length >= 2) fetchGenreValues($el)"> */}}
{{- /*   <datalist id="genre-datalist"></datalist> */}}
{{- /* </div> */}}
```

**Then regenerate:** `templ generate`

**Note:** The genre filter code is preserved in the template as a comment, making it easy to re-enable in the future if the genre field is populated.

---

## 🧪 Testing Strategy

### Integration Tests

**File:** `cmd/server/tests/tags_filter_test.go` (NEW)

**Test Structure:**
```go
package main

import (
    "testing"
    "github.com/stretchr/testify/assert"
    "github.com/stretchr/testify/require"
)

func TestTagsFilter(t *testing.T) {
    setup := setupTestServer(t)  // ✅ Using test_helpers
    defer setup.Server.Close()

    libraryID := setup.CreateLibrary(t, "Test Tags Library", "ebooks")
    _ = setup.CreateDevice(t, "Test Device", "koreader", "tags-test-123")

    client := &http.Client{}

    // Add folder to library
    addFolderToLibrary(t, setup, libraryID, "/app/uploads")

    // Helper to create book with tags
    createBook := func(title, author string, tags []string) {
        bookReq := map[string]interface{}{
            "library_id": libraryID,
            "title":      title,
            "author":     author,
            "tags":       tags,
            "file_path":  "/tmp/test.epub",
            "file_size":  1024,
            "mime_type":  "application/epub+zip",
        }
        // ... create book
    }

    // Create test books with different tags
    createBook("Foundation", "Asimov", []string{"Science Fiction", "Adventure"})
    createBook("Dune", "Herbert", []string{"Science Fiction", "Dystopian"})
    createBook("Neuromancer", "Gibson", []string{"Cyberpunk", "Science Fiction"})
    createBook("The Hobbit", "Tolkien", []string{"Fantasy", "Adventure"})

    t.Run("Tags filter - Science Fiction", func(t *testing.T) {
        req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&tags_filter=Science+Foundation", nil)
        req.Header.Set("Authorization", "Bearer "+setup.UserToken)
        // ... test implementation
        // Should return Foundation, Dune, Neuromancer (all have "Science Fiction" tag)
    })

    t.Run("Tags filter - Fantasy", func(t *testing.T) {
        // Should return only The Hobbit
    })

    t.Run("Tags autocomplete", func(t *testing.T) {
        req := httptest.NewRequest("GET", "/api/media-items/search?tags=Sci&library_id="+libraryID, nil)
        req.Header.Set("Authorization", "Bearer "+setup.UserToken)
        // Should return "Science Fiction" in autocomplete results
    })

    t.Run("Backward compatibility - genre_filter still works", func(t *testing.T) {
        // Verify genre_filter doesn't break (returns 0 results on NULL genre)
    })

    t.Run("Combined filters - tags + author", func(t *testing.T) {
        // tags_filter=Science+Fiction&author_filter=Asimov
        // Should return only Foundation
    })
}
```

### Manual QA Checklist

- [ ] Tags filter returns books with matching tags
- [ ] Tags autocomplete shows tag values as you type
- [ ] Multiple tags in book are all searchable
- [ ] Combined filters work (tags + author + series)
- [ ] Empty tags_filter returns all books (no filter)
- [ ] Backward compatibility: genre_filter still works (returns 0 on NULL)
- [ ] Browser console shows no errors
- [ ] HTMX requests complete successfully
- [ ] UI updates without page reload

---

## 📝 Documentation

### API Documentation

**File:** `docs/developer/api/media-items/search_media_items.md` (CREATE or UPDATE)

**Content:**
```markdown
# Search Media Items

## GET /api/media-items/search

Search and filter media items with fuzzy matching and tag-based filtering.

### Query Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `q` | string | No | Search query (matches title, author, series, tags) |
| `library_id` | string | No | Filter by library UUID |
| `author_filter` | string | No | Fuzzy match author field |
| `series_filter` | string | No | Fuzzy match series field |
| `tags_filter` | string | No | **Exact match tags array** (NEW) |
| `language_filter` | string | No | Fuzzy match language field |
| `year_min` | integer | No | Minimum copyright year |
| `year_max` | integer | No | Maximum copyright year |
| `has_cover` | boolean | No | Filter by cover existence |
| `sort` | string | No | Sort order (e.g., "title ASC") |
| `limit` | integer | No | Max results (default: 50) |
| `offset` | integer | No | Results offset (default: 0) |

### Tags Filter Behavior

**New in v1.x:** The `tags_filter` parameter filters books by their tags array.

- **Matching:** Exact match (not fuzzy) - the filter value must equal a tag
- **Calibre Integration:** Works seamlessly with Calibre tags (which include genre)
- **Example Tags:** `["Science Fiction", "Adventure", "Dystopian"]`

**Examples:**

```bash
# Filter by "Science Fiction" tag
GET /api/media-items/search?tags_filter=Science+Fiction

# Combine tags with author
GET /api/media-items/search?tags_filter=Science+Fiction&author_filter=Asimov

# Filter by multiple tags (returns books with ANY matching tag)
GET /api/media-items/search?tags_filter=Cyberpunk
```

### Backward Compatibility

The `genre_filter` parameter is **deprecated but still supported** for backward compatibility. It will return 0 results for books imported from Calibre (genre field is NULL).

**Migration Guide:**
```javascript
// OLD (deprecated):
genre_filter=Science+Fiction

// NEW (recommended):
tags_filter=Science+Fiction
```

### Response

**Status Codes:**
- `200 OK` - Results found
- `404 Not Found` - No results match filters
- `401 Unauthorized` - Missing or invalid JWT
- `500 Internal Server Error` - Server error

**Response Body:**
```json
[
  {
    "id": "uuid",
    "title": "Foundation",
    "author": "Isaac Asimov",
    "tags": ["Science Fiction", "Adventure"],
    "library_id": "uuid",
    "library_name": "E-Books"
  }
]
```
```

### User Documentation

**File:** `docs/user/searching-and-filtering.md` (UPDATE or CREATE)

**Content:**
```markdown
# Searching and Filtering Books

## Tag-Based Filtering

Bookhoard uses Calibre's tag system for categorization. When you import books from Calibre, their tags are automatically available for filtering.

### How Tags Work

Tags are keywords or categories assigned to books, such as:
- **Genres:** Science Fiction, Fantasy, Mystery, Romance
- **Categories:** Favorites, To Read, Read
- **Metadata:** Ebook, Owned, Borrowed

### Filtering by Tags

1. Navigate to the **Bookshelf** page
2. Use the **Tags** filter input
3. Start typing to see autocomplete suggestions
4. Select a tag or press Enter to filter

**Example:** Typing "Sci" will suggest "Science Fiction"

### Combining Filters

You can combine tags with other filters:
- **Tags + Author:** Find all Science Fiction by Asimov
- **Tags + Series:** Find all Fantasy in the "Wheel of Time" series
- **Tags + Year:** Find all Dystopian fiction published after 2000

### Tips

- Tags from Calibre are automatically imported
- You can add custom tags when editing book metadata
- Tags are case-insensitive for searching
- Autocomplete shows existing tags as you type
```

---

## 🎯 Git Commit Structure

### Commit 1: Backend SQL Layer
```
feat: add tags_filter to search query

- Add tags_filter parameter to SearchMediaItemsUnified
- Add SearchTagsValues query for autocomplete
- Keep genre_filter for backward compatibility

Relates to #<issue-number>
```

### Commit 2: Backend Service Layer
```
feat: implement tags filter in service layer

- Add TagsFilter to SearchParams struct
- Update dbParams building to include tags_filter
- Add tags case to SearchFieldValues service

Relates to #<issue-number>
```

### Commit 3: Backend Handler Layer
```
feat: add tags filter and autocomplete endpoints

- Extract tags_filter query parameter in handler
- Add tags autocomplete route
- Keep genre_filter for backward compatibility

Relates to #<issue-number>
```

### Commit 4: Frontend TypeScript
```
refactor: replace genre with tags in frontend TypeScript

- Add fetchTagValues function (keep fetchGenreValues commented out)
- Update custom-section-builder field id to "tags" (keep "genre" commented)
- Update collection-rules field mappings to "tags" (keep "genre" commented)
- Genre code preserved for easy restoration if field is populated later

Relates to #<issue-number>
```

### Commit 5: Frontend Templates
```
refactor: replace genre with tags in bookshelf UI

- Add Tags filter input (keep Genre input commented out)
- Update input name to tags_filter (keep genre_filter in comment)
- Update datalist and Alpine.js bindings
- Genre HTML preserved in template comments for future use

Relates to #<issue-number>
```

### Commit 6: Bruno Collection
```
test: update Bruno requests for tags filter

- Add Fuzzy Tags Filter.yml
- Add Field Values Search - Tags.yml
- Update Combined Search and Filters.yml

Relates to #<issue-number>
```

### Commit 7: Documentation
```
docs: document tags filter API and usage

- Add search_media_items API documentation
- Add user guide for tag-based filtering
- Document backward compatibility with genre_filter

Relates to #<issue-number>
```

### Commit 8: Integration Tests
```
test: add integration tests for tags filter

- Create tags_filter_test.go
- Test tags filter functionality
- Test tags autocomplete endpoint
- Test backward compatibility
- Test combined filters

Uses test_helpers.setupTestServer()
Relates to #<issue-number>
```

---

## ✅ Verification Checklist

### Before Completing Task

- [ ] All code compiles: `go build ./...`
- [ ] All tests pass: `go test ./... -v`
- [ ] Guidelines verified: `bash scripts/verify-guidelines.sh` (0 errors)
- [ ] Documentation renders at `/docs` endpoint
- [ ] Bruno requests execute successfully
- [ ] Manual QA completed in browser
- [ ] No regressions in existing filters
- [ ] Git commits are logical and well-structured
- [ ] Only intended changes in `git diff`

### Post-Deployment Verification

- [ ] Tags filter works in production
- [ ] Tags autocomplete returns results
- [ ] Backward compatibility maintained (genre_filter)
- [ ] No performance issues
- [ ] User feedback collected

---

## 📚 References

- **Project Guidelines:** `PROJECT_GUIDELINES.md`
- **Service Layer Pattern:** `internal/services/`
- **Test Helpers:** `cmd/server/tests/test_helpers_test.go`
- **Bruno Collection:** `bruno/media-items/`
- **API Documentation:** `docs/developer/api/`
- **User Documentation:** `docs/user/`

---

## 🚀 Next Steps

1. **Review this plan** and approve approach
2. **Implement Phase 1-3** (Backend changes)
3. **Implement Phase 4-5** (Frontend changes)
4. **Implement Phase 6-7** (Bruno + Docs)
5. **Implement Phase 8-9** (Tests + Verification)
6. **Create git commits** following commit structure
7. **Push changes** and verify in production

---

**Last Updated:** 2026-03-25
**Status:** Ready for implementation
