Update and consolidate implementation documentation
Clean up documentation by removing obsolete implementation notes and updating the Calibre OPF implementation guide. Changes: - Update CALIBRE_OPF_IMPLEMENTATION.md with namespace URL approach - Remove IMPLEMENTATION_TAGS_FILTER.md (superseded by unified search) - Remove UNIFIED_SEARCH_IMPLEMENTATION.md (implementation complete) The Calibre OPF documentation now reflects the corrected approach using full Dublin Core namespace URLs (http://purl.org/dc/elements/1.1/) instead of namespace prefixes, which were found to not work with Go's XML decoder. Documentation: #docs-cleanup
This commit is contained in:
@@ -748,7 +748,7 @@ func TestExtractCalibreSidecar(t *testing.T) {
|
|||||||
**ADD THIS INTEGRATION TEST** (optional, for comprehensive testing):
|
**ADD THIS INTEGRATION TEST** (optional, for comprehensive testing):
|
||||||
|
|
||||||
```go
|
```go
|
||||||
package tests
|
package main
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
|||||||
@@ -1,748 +0,0 @@
|
|||||||
# Implementation Plan: Replace Genre Filter with Tags Filter
|
|
||||||
|
|
||||||
## Status: 📋 Planning Phase
|
|
||||||
**Created:** 2026-03-25
|
|
||||||
**Type:** Full-Stack Task
|
|
||||||
**Priority:** Medium
|
|
||||||
**Estimated Time:** 2-3 hours
|
|
||||||
**Note:** Tags filter uses fuzzy matching (consistent with author/series filters)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎯 Overview
|
|
||||||
|
|
||||||
Replace the `genre_filter` with `tags_filter` throughout the application. The `genre` field in the database is currently NULL for all Calibre imports because Calibre uses tags for genre classification. By filtering tags instead, we leverage existing data without requiring a schema migration.
|
|
||||||
|
|
||||||
**Key Decision:** Keep `genre_filter` in the API for backward compatibility but remove it from the frontend UI.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📊 Current State Analysis
|
|
||||||
|
|
||||||
### Database State
|
|
||||||
```sql
|
|
||||||
-- Current data (from investigation):
|
|
||||||
SELECT COUNT(*) as total, COUNT(genre) as with_genre, COUNT(tags) as with_tags FROM media_items;
|
|
||||||
-- Result: 17 total books, 0 with genre, 17 with tags
|
|
||||||
```
|
|
||||||
|
|
||||||
### Problem
|
|
||||||
- **Genre field:** NULL for all imported books (Calibre doesn't populate it)
|
|
||||||
- **Tags field:** Populated with Calibre tags (including genre-like tags)
|
|
||||||
- **Filter behavior:** genre_filter returns 0 results (no matches on NULL)
|
|
||||||
- **User impact:** Cannot filter books by genre/category
|
|
||||||
|
|
||||||
### Root Cause
|
|
||||||
Calibre treats tags as genre classification. Example tags:
|
|
||||||
```
|
|
||||||
["Science Fiction", "Adventure", "Dystopian", "Favorites"]
|
|
||||||
```
|
|
||||||
|
|
||||||
The current genre filter expects a single `genre` text field, which is always NULL.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔄 Proposed Solution
|
|
||||||
|
|
||||||
### Option B: Filter Tags Instead (Selected)
|
|
||||||
|
|
||||||
**Why this approach:**
|
|
||||||
- ✅ No database migration required (zero downtime)
|
|
||||||
- ✅ Works immediately with existing Calibre data
|
|
||||||
- ✅ Low risk (isolated changes)
|
|
||||||
- ✅ Calibre users already understand this model
|
|
||||||
- ✅ Faster to implement
|
|
||||||
|
|
||||||
**Trade-offs:**
|
|
||||||
- ⚠️ Tags may contain non-genre values (e.g., "Favorites", "To Read")
|
|
||||||
- ⚠️ User filters by "Tags" instead of "Genre" in UI
|
|
||||||
|
|
||||||
**Why acceptable:**
|
|
||||||
- Calibre has the same "con" and it's not a problem there
|
|
||||||
- Users can filter their own tags as well (feature, not bug)
|
|
||||||
- API backward compatibility maintained (genre_filter still works)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📋 Implementation Checklist
|
|
||||||
|
|
||||||
### Phase 1: Backend - SQL Layer (25 minutes)
|
|
||||||
- [ ] Add `tags_filter` parameter to `SearchMediaItemsUnified` query
|
|
||||||
- [ ] Add fuzzy matching filter logic with EXISTS clause and word_similarity()
|
|
||||||
- [ ] Add tag similarity scoring to ORDER BY clause (GREATEST function)
|
|
||||||
- [ ] Keep `genre_filter` for backward compatibility
|
|
||||||
- [ ] Regenerate Go code with `sqlc generate`
|
|
||||||
- [ ] Test SQL query in database console (verify fuzzy matching works)
|
|
||||||
|
|
||||||
### Phase 2: Backend - Service Layer (20 minutes)
|
|
||||||
- [ ] Add `TagsFilter string` to `SearchParams` struct
|
|
||||||
- [ ] Update `dbParams` building to include tags_filter
|
|
||||||
- [ ] Update handler to extract `tags_filter` query parameter
|
|
||||||
- [ ] Keep existing `genre_filter` extraction (backward compat)
|
|
||||||
|
|
||||||
### Phase 3: Backend - Tags Autocomplete API (30 minutes)
|
|
||||||
- [ ] Add `SearchTagsValues` SQL query
|
|
||||||
- [ ] Add "tags" case to `SearchFieldValues` service
|
|
||||||
- [ ] Add tags route to handler (autocomplete support)
|
|
||||||
- [ ] Test tags autocomplete endpoint
|
|
||||||
|
|
||||||
### Phase 4: Frontend - TypeScript (20 minutes)
|
|
||||||
- [ ] Rename `fetchGenreValues()` → `fetchTagValues()` in bookshelf.ts
|
|
||||||
- [ ] Update field id from "genre" → "tags" in custom-section-builder.ts
|
|
||||||
- [ ] Update collection rules from "genre" → "tags" in collection-rules.ts
|
|
||||||
- [ ] No other TS files need changes (search.ts is header search only)
|
|
||||||
|
|
||||||
### Phase 5: Frontend - Templates (15 minutes)
|
|
||||||
- [ ] Update bookshelf template (genre → tags labels/inputs)
|
|
||||||
- [ ] Change datalist id from "genre-datalist" → "tags-datalist"
|
|
||||||
- [ ] Update @input handler from `fetchGenreValues` → `fetchTagValues`
|
|
||||||
- [ ] Regenerate template Go files with `templ generate`
|
|
||||||
|
|
||||||
### Phase 6: Bruno Collection Updates (15 minutes)
|
|
||||||
- [ ] Create `Fuzzy Tags Filter.yml` (test fuzzy matching behavior)
|
|
||||||
- [ ] Create `Field Values Search - Tags.yml` (test autocomplete)
|
|
||||||
- [ ] Update `Combined Search and Filters.yml` (genre → tags_filter)
|
|
||||||
- [ ] Add test cases for fuzzy matching (e.g., "Sci Fi" → "Science Fiction")
|
|
||||||
- [ ] Add documentation for fuzzy tags filter behavior
|
|
||||||
|
|
||||||
### Phase 7: Documentation (20 minutes)
|
|
||||||
- [ ] Create `docs/developer/api/media-items/search_media_items.md` (or update existing)
|
|
||||||
- [ ] Document tags_filter parameter
|
|
||||||
- [ ] Document backward compatibility with genre_filter
|
|
||||||
- [ ] Add examples of tags filtering
|
|
||||||
- [ ] Update any relevant user documentation
|
|
||||||
|
|
||||||
### Phase 8: Integration Tests (30 minutes)
|
|
||||||
- [ ] Create `tags_filter_test.go` in `cmd/server/tests/`
|
|
||||||
- [ ] Test with `setupTestServer()` helper from `test_helpers_test.go`
|
|
||||||
- [ ] Test tags filter returns matching books
|
|
||||||
- [ ] Test tags autocomplete endpoint
|
|
||||||
- [ ] Test backward compatibility (genre_filter still works)
|
|
||||||
- [ ] Test combined filters (tags + author + series)
|
|
||||||
|
|
||||||
### Phase 9: Verification & Testing (20 minutes)
|
|
||||||
- [ ] Run full test suite: `go test ./... -v`
|
|
||||||
- [ ] Test tags filter in browser (manual QA)
|
|
||||||
- [ ] Verify tags autocomplete works
|
|
||||||
- [ ] Verify backward compatibility (genre_filter)
|
|
||||||
- [ ] Check no regressions in existing filters
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔧 Technical Changes
|
|
||||||
|
|
||||||
### 1. SQL Query Changes
|
|
||||||
|
|
||||||
**File:** `internal/database/queries/queries.sql`
|
|
||||||
|
|
||||||
**Location:** Line 435-437 (SearchMediaItemsUnified query)
|
|
||||||
|
|
||||||
**Change:**
|
|
||||||
```sql
|
|
||||||
-- BEFORE:
|
|
||||||
-- Fuzzy genre filter
|
|
||||||
AND (sqlc.narg('genre_filter') = '' OR word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')) > 0.3)
|
|
||||||
|
|
||||||
-- AFTER:
|
|
||||||
-- Fuzzy genre filter (kept for backward compatibility)
|
|
||||||
AND (sqlc.narg('genre_filter') = '' OR word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')) > 0.3)
|
|
||||||
-- Tags filter (NEW - fuzzy match against tags array)
|
|
||||||
AND (sqlc.narg('tags_filter') = '' OR EXISTS (
|
|
||||||
SELECT 1 FROM unnest(mi.tags_search) AS tag
|
|
||||||
WHERE word_similarity(sqlc.narg('tags_filter'), tag) > 0.3
|
|
||||||
))
|
|
||||||
```
|
|
||||||
|
|
||||||
**New Query:** Add `SearchTagsValues` for autocomplete
|
|
||||||
```sql
|
|
||||||
-- name: SearchTagsValues :many
|
|
||||||
SELECT
|
|
||||||
unnest(mi.tags_search) as value,
|
|
||||||
COUNT(*) as count,
|
|
||||||
word_similarity(sqlc.narg('search_query'), unnest(mi.tags_search))::float8 as score
|
|
||||||
FROM media_items mi
|
|
||||||
JOIN libraries l ON mi.library_id = l.id
|
|
||||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
|
||||||
WHERE COALESCE(lv.is_visible, true) = true
|
|
||||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
|
||||||
AND unnest(mi.tags_search) IS NOT NULL
|
|
||||||
AND word_similarity(sqlc.narg('search_query'), unnest(mi.tags_search)) > 0.3
|
|
||||||
GROUP BY unnest(mi.tags_search), word_similarity(sqlc.narg('search_query'), unnest(mi.tags_search))::float8
|
|
||||||
ORDER BY word_similarity(sqlc.narg('search_query'), unnest(mi.tags_search))::float8 DESC, COUNT DESC
|
|
||||||
LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset');
|
|
||||||
```
|
|
||||||
|
|
||||||
**ORDER BY Clause Update (line ~484):**
|
|
||||||
Add tag similarity scoring to the GREATEST() calculation:
|
|
||||||
```sql
|
|
||||||
-- In the ORDER BY clause, add to the GREATEST() function:
|
|
||||||
GREATEST(
|
|
||||||
CASE WHEN sqlc.narg('is_exact_search') = false THEN
|
|
||||||
word_similarity(sqlc.narg('search_query'), mi.title)
|
|
||||||
ELSE 0 END,
|
|
||||||
CASE WHEN sqlc.narg('is_exact_search') = false THEN
|
|
||||||
word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, ''))
|
|
||||||
ELSE 0 END,
|
|
||||||
word_similarity(sqlc.narg('author_filter'), COALESCE(mi.author, '')),
|
|
||||||
word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')),
|
|
||||||
(SELECT MAX(word_similarity(sqlc.narg('tags_filter'), tag))
|
|
||||||
FROM unnest(mi.tags_search) AS tag) -- NEW: Tag similarity score
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
This ensures results with better tag matches appear first.
|
|
||||||
|
|
||||||
### 2. Service Layer Changes
|
|
||||||
|
|
||||||
**File:** `internal/services/search.go`
|
|
||||||
|
|
||||||
**Change 1:** Update SearchParams struct (line 22-36)
|
|
||||||
```go
|
|
||||||
type SearchParams struct {
|
|
||||||
UserID pgtype.UUID
|
|
||||||
LibraryID pgtype.UUID
|
|
||||||
AuthorFilter string
|
|
||||||
SeriesFilter string
|
|
||||||
GenreFilter string // KEPT for backward compatibility
|
|
||||||
TagsFilter string // NEW
|
|
||||||
LanguageFilter string
|
|
||||||
YearMin int
|
|
||||||
YearMax int
|
|
||||||
HasCover bool
|
|
||||||
SearchQuery string
|
|
||||||
Sort string
|
|
||||||
Limit int
|
|
||||||
Offset int
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Change 2:** Update dbParams building (line 67-82)
|
|
||||||
```go
|
|
||||||
dbParams := database.SearchMediaItemsUnifiedParams{
|
|
||||||
UserID: params.UserID,
|
|
||||||
LibraryID: params.LibraryID,
|
|
||||||
AuthorFilter: pgtype.Text{String: params.AuthorFilter, Valid: params.AuthorFilter != ""},
|
|
||||||
SeriesFilter: pgtype.Text{String: params.SeriesFilter, Valid: params.SeriesFilter != ""},
|
|
||||||
GenreFilter: pgtype.Text{String: params.GenreFilter, Valid: params.GenreFilter != ""}, // KEPT
|
|
||||||
TagsFilter: pgtype.Text{String: params.TagsFilter, Valid: params.TagsFilter != ""}, // NEW
|
|
||||||
LanguageFilter: pgtype.Text{String: params.LanguageFilter, Valid: params.LanguageFilter != ""},
|
|
||||||
// ... rest of params
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Change 3:** Add tags case to SearchFieldValues (after line 151)
|
|
||||||
```go
|
|
||||||
case "tags":
|
|
||||||
results, err := s.db.SearchTagsValues(ctx, database.SearchTagsValuesParams{
|
|
||||||
SearchQuery: pgtype.Text{String: params.SearchQuery, Valid: true},
|
|
||||||
UserID: params.UserID,
|
|
||||||
LibraryID: params.LibraryID,
|
|
||||||
Limit: pgtype.Int4{Int32: int32(params.Limit), Valid: true},
|
|
||||||
Offset: pgtype.Int4{Int32: int32(params.Offset), Valid: true},
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
fieldValues := make([]FieldValue, len(results))
|
|
||||||
for i, r := range results {
|
|
||||||
fieldValues[i] = FieldValue{Value: r.Value.String, Count: r.Count, Score: r.Score}
|
|
||||||
}
|
|
||||||
return fieldValues, nil
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Handler Changes
|
|
||||||
|
|
||||||
**File:** `internal/handlers/media.go`
|
|
||||||
|
|
||||||
**Change 1:** Extract tags_filter parameter (after line 1414)
|
|
||||||
```go
|
|
||||||
// Extract filter parameters
|
|
||||||
authorFilter := c.QueryParam("author_filter")
|
|
||||||
seriesFilter := c.QueryParam("series_filter")
|
|
||||||
genreFilter := c.QueryParam("genre_filter") // KEPT for backward compatibility
|
|
||||||
tagsFilter := c.QueryParam("tags_filter") // NEW
|
|
||||||
languageFilter := c.QueryParam("language_filter")
|
|
||||||
```
|
|
||||||
|
|
||||||
**Change 2:** Add to SearchParams (line 1426-1440)
|
|
||||||
```go
|
|
||||||
params := services.SearchParams{
|
|
||||||
UserID: userID.ID,
|
|
||||||
LibraryID: libUUID,
|
|
||||||
SearchQuery: query,
|
|
||||||
AuthorFilter: authorFilter,
|
|
||||||
SeriesFilter: seriesFilter,
|
|
||||||
GenreFilter: genreFilter, // KEPT
|
|
||||||
TagsFilter: tagsFilter, // NEW
|
|
||||||
LanguageFilter: languageFilter,
|
|
||||||
// ... rest of params
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Change 3:** Add tags autocomplete route (after line 1379)
|
|
||||||
```go
|
|
||||||
if tags := c.QueryParam("tags"); tags != "" {
|
|
||||||
return mh.handleFieldValuesSearch(c, userID.ID, "tags", tags)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Frontend TypeScript Changes
|
|
||||||
|
|
||||||
**File:** `web/src/bookshelf.ts`
|
|
||||||
|
|
||||||
**Change:** Line 297-300
|
|
||||||
```typescript
|
|
||||||
// AFTER (add new function, keep old commented out):
|
|
||||||
async fetchTagValues(input: HTMLInputElement): Promise<void> {
|
|
||||||
await this.fetchFieldValues("tags", input.value, "tags-datalist");
|
|
||||||
}
|
|
||||||
|
|
||||||
// TODO: Uncomment if genre field is populated in future
|
|
||||||
// async fetchGenreValues(input: HTMLInputElement): Promise<void> {
|
|
||||||
// await this.fetchFieldValues("genre", input.value, "genre-datalist");
|
|
||||||
// }
|
|
||||||
```
|
|
||||||
|
|
||||||
**File:** `web/src/custom-section-builder.ts`
|
|
||||||
|
|
||||||
**Change:** Line 49-50
|
|
||||||
```typescript
|
|
||||||
// AFTER (add new field, keep old commented out):
|
|
||||||
[
|
|
||||||
{ id: "tags", label: "Tags" },
|
|
||||||
// TODO: Uncomment if genre field is populated in future
|
|
||||||
// { id: "genre", label: "Genre" },
|
|
||||||
// ... other fields
|
|
||||||
]
|
|
||||||
```
|
|
||||||
|
|
||||||
### 5. Template Changes
|
|
||||||
|
|
||||||
**File:** `templates/bookshelf.templ` (source file)
|
|
||||||
|
|
||||||
**Locate:** Find the genre filter input section (around line 131 in generated Go file)
|
|
||||||
|
|
||||||
**Change:**
|
|
||||||
```html
|
|
||||||
<!-- AFTER: Add tags filter, comment out genre filter for future use -->
|
|
||||||
<!-- Tags Filter with Autocomplete -->
|
|
||||||
<div class="flex-1 min-w-[150px]">
|
|
||||||
<label class="block text-sm font-medium mb-2" style="color: var(--text-secondary)">Tags</label>
|
|
||||||
<input type="text" name="tags_filter" placeholder="Filter by tags"
|
|
||||||
class="w-full px-3 py-2 border rounded-lg"
|
|
||||||
style="background-color: var(--bg-primary); color: var(--text-primary); border-color: var(--border);"
|
|
||||||
hx-get="/api/media-items/search" hx-target="#books-grid"
|
|
||||||
hx-trigger="keyup[key=='Enter'] from:#filter-form" hx-include="#filter-form"
|
|
||||||
list="tags-datalist" @input.debounce.300ms="if($el.value.length >= 2) fetchTagValues($el)">
|
|
||||||
<datalist id="tags-datalist"></datalist>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- TODO: Uncomment if genre field is populated in future -->
|
|
||||||
{{- /* Genre Filter with Autocomplete */}
|
|
||||||
{{- /* <div class="flex-1 min-w-[150px]"> */}
|
|
||||||
{{- /* <label class="block text-sm font-medium mb-2" style="color: var(--text-secondary)">Genre</label> */}
|
|
||||||
{{- /* <input type="text" name="genre_filter" placeholder="Filter by genre" */}}
|
|
||||||
{{- /* class="w-full px-3 py-2 border rounded-lg" */}}
|
|
||||||
{{- /* style="background-color: var(--bg-primary); color: var(--text-primary); border-color: var(--border);" */}}
|
|
||||||
{{- /* hx-get="/api/media-items/search" hx-target="#books-grid" */}}
|
|
||||||
{{- /* hx-trigger="keyup[key=='Enter'] from:#filter-form" hx-include="#filter-form" */}}
|
|
||||||
{{- /* list="genre-datalist" @input.debounce.300ms="if($el.value.length >= 2) fetchGenreValues($el)"> */}}
|
|
||||||
{{- /* <datalist id="genre-datalist"></datalist> */}}
|
|
||||||
{{- /* </div> */}}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Then regenerate:** `templ generate`
|
|
||||||
|
|
||||||
**Note:** The genre filter code is preserved in the template as a comment, making it easy to re-enable in the future if the genre field is populated.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🧪 Testing Strategy
|
|
||||||
|
|
||||||
### Integration Tests
|
|
||||||
|
|
||||||
**File:** `cmd/server/tests/tags_filter_test.go` (NEW)
|
|
||||||
|
|
||||||
**Test Structure:**
|
|
||||||
```go
|
|
||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"testing"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestTagsFilter(t *testing.T) {
|
|
||||||
setup := setupDeviceTest(t) // ✅ Using test_helpers
|
|
||||||
defer setup.Server.Close()
|
|
||||||
|
|
||||||
libraryID := setup.CreateLibrary(t, "Test Tags Library", "ebooks")
|
|
||||||
_ = setup.CreateDevice(t, "Test Device", "koreader", "tags-test-123")
|
|
||||||
|
|
||||||
client := &http.Client{}
|
|
||||||
|
|
||||||
// Add folder to library
|
|
||||||
addFolderToLibrary(t, setup, libraryID, "/app/uploads")
|
|
||||||
|
|
||||||
// Helper to create book with tags
|
|
||||||
createBook := func(title, author string, tags []string) {
|
|
||||||
bookReq := map[string]interface{}{
|
|
||||||
"library_id": libraryID,
|
|
||||||
"title": title,
|
|
||||||
"author": author,
|
|
||||||
"tags": tags,
|
|
||||||
"file_path": "/tmp/test.epub",
|
|
||||||
"file_size": 1024,
|
|
||||||
"mime_type": "application/epub+zip",
|
|
||||||
}
|
|
||||||
// ... create book
|
|
||||||
}
|
|
||||||
|
|
||||||
// Create test books with different tags
|
|
||||||
createBook("Foundation", "Asimov", []string{"Science Fiction", "Adventure"})
|
|
||||||
createBook("Dune", "Herbert", []string{"Science Fiction", "Dystopian"})
|
|
||||||
createBook("Neuromancer", "Gibson", []string{"Cyberpunk", "Science Fiction"})
|
|
||||||
createBook("The Hobbit", "Tolkien", []string{"Fantasy", "Adventure"})
|
|
||||||
|
|
||||||
t.Run("Tags filter - Science Fiction", func(t *testing.T) {
|
|
||||||
req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&tags_filter=Science+Fiction", nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+setup.UserToken)
|
|
||||||
// ... test implementation
|
|
||||||
// Should return Foundation, Dune, Neuromancer (all have "Science Fiction" tag)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Tags filter - fuzzy match", func(t *testing.T) {
|
|
||||||
req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&tags_filter=Sci+Fi", nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+setup.UserToken)
|
|
||||||
// ... test implementation
|
|
||||||
// Should return Foundation, Dune, Neuromancer (fuzzy matches "Science Fiction")
|
|
||||||
// Results should be ranked by similarity score
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Tags filter - Fantasy", func(t *testing.T) {
|
|
||||||
// Should return only The Hobbit
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Tags autocomplete", func(t *testing.T) {
|
|
||||||
req := httptest.NewRequest("GET", "/api/media-items/search?tags=Sci&library_id="+libraryID, nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+setup.UserToken)
|
|
||||||
// Should return "Science Fiction" in autocomplete results with fuzzy matching
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Backward compatibility - genre_filter still works", func(t *testing.T) {
|
|
||||||
// Verify genre_filter doesn't break (returns 0 results on NULL genre)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Combined filters - tags + author", func(t *testing.T) {
|
|
||||||
// tags_filter=Sci+Fi&author_filter=Asimov
|
|
||||||
// Should return only Foundation
|
|
||||||
})
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Manual QA Checklist
|
|
||||||
|
|
||||||
- [ ] Tags filter returns books with matching tags
|
|
||||||
- [ ] Tags filter uses fuzzy matching (e.g., "Sci Fi" matches "Science Fiction")
|
|
||||||
- [ ] Results are ranked by tag similarity score (best matches first)
|
|
||||||
- [ ] Tags autocomplete shows tag values as you type
|
|
||||||
- [ ] Multiple tags in book are all searchable
|
|
||||||
- [ ] Combined filters work (tags + author + series)
|
|
||||||
- [ ] Empty tags_filter returns all books (no filter)
|
|
||||||
- [ ] Backward compatibility: genre_filter still works (returns 0 on NULL)
|
|
||||||
- [ ] Browser console shows no errors
|
|
||||||
- [ ] HTMX requests complete successfully
|
|
||||||
- [ ] UI updates without page reload
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📝 Documentation
|
|
||||||
|
|
||||||
### API Documentation
|
|
||||||
|
|
||||||
**File:** `docs/developer/api/media-items/search_media_items.md` (CREATE or UPDATE)
|
|
||||||
|
|
||||||
**Content:**
|
|
||||||
```markdown
|
|
||||||
# Search Media Items
|
|
||||||
|
|
||||||
## GET /api/media-items/search
|
|
||||||
|
|
||||||
Search and filter media items with fuzzy matching and tag-based filtering.
|
|
||||||
|
|
||||||
### Query Parameters
|
|
||||||
|
|
||||||
| Parameter | Type | Required | Description |
|
|
||||||
|-----------|------|----------|-------------|
|
|
||||||
| `q` | string | No | Search query (matches title, author, series, tags) |
|
|
||||||
| `library_id` | string | No | Filter by library UUID |
|
|
||||||
| `author_filter` | string | No | Fuzzy match author field |
|
|
||||||
| `series_filter` | string | No | Fuzzy match series field |
|
|
||||||
| `tags_filter` | string | No | **Fuzzy match tags array** (NEW) |
|
|
||||||
| `language_filter` | string | No | Fuzzy match language field |
|
|
||||||
| `year_min` | integer | No | Minimum copyright year |
|
|
||||||
| `year_max` | integer | No | Maximum copyright year |
|
|
||||||
| `has_cover` | boolean | No | Filter by cover existence |
|
|
||||||
| `sort` | string | No | Sort order (e.g., "title ASC") |
|
|
||||||
| `limit` | integer | No | Max results (default: 50) |
|
|
||||||
| `offset` | integer | No | Results offset (default: 0) |
|
|
||||||
|
|
||||||
### Tags Filter Behavior
|
|
||||||
|
|
||||||
**New in v1.x:** The `tags_filter` parameter filters books by their tags array.
|
|
||||||
|
|
||||||
- **Matching:** Fuzzy match using `word_similarity()` - matches tags with similarity > 0.3
|
|
||||||
- **Calibre Integration:** Works seamlessly with Calibre tags (which include genre)
|
|
||||||
- **Result Ranking:** Best tag matches appear first in results
|
|
||||||
- **Example Tags:** `["Science Fiction", "Adventure", "Dystopian"]`
|
|
||||||
|
|
||||||
**Fuzzy Matching Examples:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Exact match: "Science Fiction"
|
|
||||||
GET /api/media-items/search?tags_filter=Science+Fiction
|
|
||||||
# Returns: All books with "Science Fiction" tag
|
|
||||||
|
|
||||||
# Fuzzy match: "Sci Fi" → "Science Fiction"
|
|
||||||
GET /api/media-items/search?tags_filter=Sci+Fi
|
|
||||||
# Returns: Books with "Science Fiction", "Sci-Fi", "Scifi" (score > 0.3)
|
|
||||||
|
|
||||||
# Combine tags with author
|
|
||||||
GET /api/media-items/search?tags_filter=Sci+Fi&author_filter=Asimov
|
|
||||||
# Returns: Science Fiction books by Asimov, best matches first
|
|
||||||
|
|
||||||
# Fuzzy match with variations
|
|
||||||
GET /api/media-items/search?tags_filter=Cyber
|
|
||||||
# Returns: Books with "Cyberpunk", "Cyber", "Cyberthriller" tags
|
|
||||||
```
|
|
||||||
|
|
||||||
### Backward Compatibility
|
|
||||||
|
|
||||||
The `genre_filter` parameter is **deprecated but still supported** for backward compatibility. It will return 0 results for books imported from Calibre (genre field is NULL).
|
|
||||||
|
|
||||||
**Migration Guide:**
|
|
||||||
```javascript
|
|
||||||
// OLD (deprecated):
|
|
||||||
genre_filter=Science+Fiction
|
|
||||||
|
|
||||||
// NEW (recommended):
|
|
||||||
tags_filter=Science+Fiction
|
|
||||||
```
|
|
||||||
|
|
||||||
### Response
|
|
||||||
|
|
||||||
**Status Codes:**
|
|
||||||
- `200 OK` - Results found
|
|
||||||
- `404 Not Found` - No results match filters
|
|
||||||
- `401 Unauthorized` - Missing or invalid JWT
|
|
||||||
- `500 Internal Server Error` - Server error
|
|
||||||
|
|
||||||
**Response Body:**
|
|
||||||
```json
|
|
||||||
[
|
|
||||||
{
|
|
||||||
"id": "uuid",
|
|
||||||
"title": "Foundation",
|
|
||||||
"author": "Isaac Asimov",
|
|
||||||
"tags": ["Science Fiction", "Adventure"],
|
|
||||||
"library_id": "uuid",
|
|
||||||
"library_name": "E-Books"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
```
|
|
||||||
|
|
||||||
### User Documentation
|
|
||||||
|
|
||||||
**File:** `docs/user/searching-and-filtering.md` (UPDATE or CREATE)
|
|
||||||
|
|
||||||
**Content:**
|
|
||||||
```markdown
|
|
||||||
# Searching and Filtering Books
|
|
||||||
|
|
||||||
## Tag-Based Filtering
|
|
||||||
|
|
||||||
Bookhoard uses Calibre's tag system for categorization. When you import books from Calibre, their tags are automatically available for filtering.
|
|
||||||
|
|
||||||
### How Tags Work
|
|
||||||
|
|
||||||
Tags are keywords or categories assigned to books, such as:
|
|
||||||
- **Genres:** Science Fiction, Fantasy, Mystery, Romance
|
|
||||||
- **Categories:** Favorites, To Read, Read
|
|
||||||
- **Metadata:** Ebook, Owned, Borrowed
|
|
||||||
|
|
||||||
### Filtering by Tags
|
|
||||||
|
|
||||||
1. Navigate to the **Bookshelf** page
|
|
||||||
2. Use the **Tags** filter input
|
|
||||||
3. Start typing to see autocomplete suggestions
|
|
||||||
4. Select a tag or press Enter to filter
|
|
||||||
|
|
||||||
**Example:** Typing "Sci" will suggest "Science Fiction"
|
|
||||||
|
|
||||||
### Combining Filters
|
|
||||||
|
|
||||||
You can combine tags with other filters:
|
|
||||||
- **Tags + Author:** Find all Science Fiction by Asimov
|
|
||||||
- **Tags + Series:** Find all Fantasy in the "Wheel of Time" series
|
|
||||||
- **Tags + Year:** Find all Dystopian fiction published after 2000
|
|
||||||
|
|
||||||
### Tips
|
|
||||||
|
|
||||||
- Tags from Calibre are automatically imported
|
|
||||||
- You can add custom tags when editing book metadata
|
|
||||||
- Tags use fuzzy matching - "Sci Fi" will match "Science Fiction"
|
|
||||||
- Best matching tags appear first in results
|
|
||||||
- Autocomplete shows existing tags as you type with relevance scores
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎯 Git Commit Structure
|
|
||||||
|
|
||||||
### Commit 1: Backend SQL Layer
|
|
||||||
```
|
|
||||||
feat: add fuzzy tags_filter to search query
|
|
||||||
|
|
||||||
- Add fuzzy tags_filter parameter to SearchMediaItemsUnified
|
|
||||||
- Add EXISTS clause with word_similarity() for fuzzy tag matching
|
|
||||||
- Add tag similarity scoring to ORDER BY clause
|
|
||||||
- Add SearchTagsValues query for autocomplete
|
|
||||||
- Keep genre_filter for backward compatibility
|
|
||||||
|
|
||||||
Relates to #<issue-number>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Commit 2: Backend Service Layer
|
|
||||||
```
|
|
||||||
feat: implement tags filter in service layer
|
|
||||||
|
|
||||||
- Add TagsFilter to SearchParams struct
|
|
||||||
- Update dbParams building to include tags_filter
|
|
||||||
- Add tags case to SearchFieldValues service
|
|
||||||
|
|
||||||
Relates to #<issue-number>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Commit 3: Backend Handler Layer
|
|
||||||
```
|
|
||||||
feat: add tags filter and autocomplete endpoints
|
|
||||||
|
|
||||||
- Extract tags_filter query parameter in handler
|
|
||||||
- Add tags autocomplete route
|
|
||||||
- Keep genre_filter for backward compatibility
|
|
||||||
|
|
||||||
Relates to #<issue-number>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Commit 4: Frontend TypeScript
|
|
||||||
```
|
|
||||||
refactor: replace genre with tags in frontend TypeScript
|
|
||||||
|
|
||||||
- Add fetchTagValues function (keep fetchGenreValues commented out)
|
|
||||||
- Update custom-section-builder field id to "tags" (keep "genre" commented)
|
|
||||||
- Update collection-rules field mappings to "tags" (keep "genre" commented)
|
|
||||||
- Genre code preserved for easy restoration if field is populated later
|
|
||||||
|
|
||||||
Relates to #<issue-number>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Commit 5: Frontend Templates
|
|
||||||
```
|
|
||||||
refactor: replace genre with tags in bookshelf UI
|
|
||||||
|
|
||||||
- Add Tags filter input (keep Genre input commented out)
|
|
||||||
- Update input name to tags_filter (keep genre_filter in comment)
|
|
||||||
- Update datalist and Alpine.js bindings
|
|
||||||
- Genre HTML preserved in template comments for future use
|
|
||||||
|
|
||||||
Relates to #<issue-number>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Commit 6: Bruno Collection
|
|
||||||
```
|
|
||||||
test: update Bruno requests for tags filter
|
|
||||||
|
|
||||||
- Add Fuzzy Tags Filter.yml
|
|
||||||
- Add Field Values Search - Tags.yml
|
|
||||||
- Update Combined Search and Filters.yml
|
|
||||||
|
|
||||||
Relates to #<issue-number>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Commit 7: Documentation
|
|
||||||
```
|
|
||||||
docs: document tags filter API and usage
|
|
||||||
|
|
||||||
- Add search_media_items API documentation
|
|
||||||
- Add user guide for tag-based filtering
|
|
||||||
- Document backward compatibility with genre_filter
|
|
||||||
|
|
||||||
Relates to #<issue-number>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Commit 8: Integration Tests
|
|
||||||
```
|
|
||||||
test: add integration tests for tags filter
|
|
||||||
|
|
||||||
- Create tags_filter_test.go
|
|
||||||
- Test tags filter functionality
|
|
||||||
- Test tags autocomplete endpoint
|
|
||||||
- Test backward compatibility
|
|
||||||
- Test combined filters
|
|
||||||
|
|
||||||
Uses test_helpers.setupTestServer()
|
|
||||||
Relates to #<issue-number>
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ✅ Verification Checklist
|
|
||||||
|
|
||||||
### Before Completing Task
|
|
||||||
|
|
||||||
- [ ] All code compiles: `go build ./...`
|
|
||||||
- [ ] All tests pass: `go test ./... -v`
|
|
||||||
- [ ] Guidelines verified: `bash scripts/verify-guidelines.sh` (0 errors)
|
|
||||||
- [ ] Documentation renders at `/docs` endpoint
|
|
||||||
- [ ] Bruno requests execute successfully
|
|
||||||
- [ ] Manual QA completed in browser
|
|
||||||
- [ ] No regressions in existing filters
|
|
||||||
- [ ] Git commits are logical and well-structured
|
|
||||||
- [ ] Only intended changes in `git diff`
|
|
||||||
|
|
||||||
### Post-Deployment Verification
|
|
||||||
|
|
||||||
- [ ] Tags filter works in production
|
|
||||||
- [ ] Tags autocomplete returns results
|
|
||||||
- [ ] Backward compatibility maintained (genre_filter)
|
|
||||||
- [ ] No performance issues
|
|
||||||
- [ ] User feedback collected
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📚 References
|
|
||||||
|
|
||||||
- **Project Guidelines:** `PROJECT_GUIDELINES.md`
|
|
||||||
- **Service Layer Pattern:** `internal/services/`
|
|
||||||
- **Test Helpers:** `cmd/server/tests/test_helpers_test.go`
|
|
||||||
- **Bruno Collection:** `bruno/media-items/`
|
|
||||||
- **API Documentation:** `docs/developer/api/`
|
|
||||||
- **User Documentation:** `docs/user/`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚀 Next Steps
|
|
||||||
|
|
||||||
1. **Review this plan** and approve approach
|
|
||||||
2. **Implement Phase 1-3** (Backend changes)
|
|
||||||
3. **Implement Phase 4-5** (Frontend changes)
|
|
||||||
4. **Implement Phase 6-7** (Bruno + Docs)
|
|
||||||
5. **Implement Phase 8-9** (Tests + Verification)
|
|
||||||
6. **Create git commits** following commit structure
|
|
||||||
7. **Push changes** and verify in production
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Last Updated:** 2026-03-25
|
|
||||||
**Status:** Ready for implementation
|
|
||||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user