Bookhoard/bookhoard
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 12 Wiki Activity

docs: update implementation plan with fuzzy matching decision

Browse Source 
- Update SQL queries to use fuzzy matching for tags_filter
- Add ORDER BY clause changes for tag similarity scoring
- Update test code to use setupDeviceTest() instead of setupTestServer()
- Document fuzzy matching behavior throughout
- Update examples to show fuzzy matching ("Sci Fi" → "Science Fiction")
- Add missing comma fix to SQL ORDER BY clause
- Correct test helper function references
- Note that collection-rules.ts already supports both genre and tags

Updates the implementation plan to reflect the decision to use fuzzy
matching for tags_filter, making it consistent with other filters.
Includes corrections to test code and documentation improvements.

Relates to IMPLEMENTATION_TAGS_FILTER.md planning updates
This commit is contained in:
john-okeefe
2026-03-25 20:38:40 -04:00
parent fbb0023621
commit a64f14047d
1 changed files with 70 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
IMPLEMENTATION_TAGS_FILTER.md
+70 -36
View File
@@ -5,6 +5,7 @@
**Type:** Full-Stack Task **Type:** Full-Stack Task
**Priority:** Medium **Priority:** Medium
**Estimated Time:** 2-3 hours **Estimated Time:** 2-3 hours
**Note:** Tags filter uses fuzzy matching (consistent with author/series filters)
--- ---
@@ -65,11 +66,13 @@ The current genre filter expects a single `genre` text field, which is always NU
## 📋 Implementation Checklist ## 📋 Implementation Checklist
### Phase 1: Backend - SQL Layer (15 minutes) ### Phase 1: Backend - SQL Layer (25 minutes)
- [ ] Add `tags_filter` parameter to `SearchMediaItemsUnified` query - [ ] Add `tags_filter` parameter to `SearchMediaItemsUnified` query
- [ ] Update filter logic from `word_similarity()` to `= ANY()` for array matching - [ ] Add fuzzy matching filter logic with EXISTS clause and word_similarity()
- [ ] Add tag similarity scoring to ORDER BY clause (GREATEST function)
- [ ] Keep `genre_filter` for backward compatibility - [ ] Keep `genre_filter` for backward compatibility
- [ ] Regenerate Go code with `sqlc generate` - [ ] Regenerate Go code with `sqlc generate`
- [ ] Test SQL query in database console (verify fuzzy matching works)
### Phase 2: Backend - Service Layer (20 minutes) ### Phase 2: Backend - Service Layer (20 minutes)
- [ ] Add `TagsFilter string` to `SearchParams` struct - [ ] Add `TagsFilter string` to `SearchParams` struct
@@ -96,10 +99,11 @@ The current genre filter expects a single `genre` text field, which is always NU
- [ ] Regenerate template Go files with `templ generate` - [ ] Regenerate template Go files with `templ generate`
### Phase 6: Bruno Collection Updates (15 minutes) ### Phase 6: Bruno Collection Updates (15 minutes)
- [ ] Create `Fuzzy Tags Filter.yml` (rename/update genre filter test) - [ ] Create `Fuzzy Tags Filter.yml` (test fuzzy matching behavior)
- [ ] Create `Field Values Search - Tags.yml` (rename genre autocomplete) - [ ] Create `Field Values Search - Tags.yml` (test autocomplete)
- [ ] Update `Combined Search and Filters.yml` (genre → tags_filter) - [ ] Update `Combined Search and Filters.yml` (genre → tags_filter)
- [ ] Add documentation for tags filter behavior - [ ] Add test cases for fuzzy matching (e.g., "Sci Fi" → "Science Fiction")
- [ ] Add documentation for fuzzy tags filter behavior
### Phase 7: Documentation (20 minutes) ### Phase 7: Documentation (20 minutes)
- [ ] Create `docs/developer/api/media-items/search_media_items.md` (or update existing) - [ ] Create `docs/developer/api/media-items/search_media_items.md` (or update existing)
@@ -142,8 +146,11 @@ AND (sqlc.narg('genre_filter') = '' OR word_similarity(sqlc.narg('genre_filter')
-- AFTER: -- AFTER:
-- Fuzzy genre filter (kept for backward compatibility) -- Fuzzy genre filter (kept for backward compatibility)
AND (sqlc.narg('genre_filter') = '' OR word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')) > 0.3) AND (sqlc.narg('genre_filter') = '' OR word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')) > 0.3)
-- Tags filter (NEW - matches against tags array) -- Tags filter (NEW - fuzzy match against tags array)
AND (sqlc.narg('tags_filter') = '' OR sqlc.narg('tags_filter') = ANY(mi.tags_search)) AND (sqlc.narg('tags_filter') = '' OR EXISTS (
SELECT 1 FROM unnest(mi.tags_search) AS tag
WHERE word_similarity(sqlc.narg('tags_filter'), tag) > 0.3
))
``` ```
**New Query:** Add `SearchTagsValues` for autocomplete **New Query:** Add `SearchTagsValues` for autocomplete
@@ -165,6 +172,26 @@ ORDER BY word_similarity(sqlc.narg('search_query'), unnest(mi.tags_search))::flo
LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset'); LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset');
``` ```
**ORDER BY Clause Update (line ~484):**
Add tag similarity scoring to the GREATEST() calculation:
```sql
-- In the ORDER BY clause, add to the GREATEST() function:
GREATEST(
CASE WHEN sqlc.narg('is_exact_search') = false THEN
word_similarity(sqlc.narg('search_query'), mi.title)
ELSE 0 END,
CASE WHEN sqlc.narg('is_exact_search') = false THEN
word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, ''))
ELSE 0 END,
word_similarity(sqlc.narg('author_filter'), COALESCE(mi.author, '')),
word_similarity(sqlc.narg('genre_filter'), COALESCE(mi.genre, '')),
(SELECT MAX(word_similarity(sqlc.narg('tags_filter'), tag))
FROM unnest(mi.tags_search) AS tag) -- NEW: Tag similarity score
)
```
This ensures results with better tag matches appear first.
### 2. Service Layer Changes ### 2. Service Layer Changes
**File:** `internal/services/search.go` **File:** `internal/services/search.go`
@@ -289,19 +316,6 @@ async fetchTagValues(input: HTMLInputElement): Promise<void> {
] ]
``` ```
**File:** `web/src/collection-rules.ts`
**Change:** Line 45
```typescript
// AFTER (add new field, keep old commented out):
const ruleFields = {
tags: "Tags",
// TODO: Uncomment if genre field is populated in future
// genre: "Genre",
// ... other fields
};
```
### 5. Template Changes ### 5. Template Changes
**File:** `templates/bookshelf.templ` (source file) **File:** `templates/bookshelf.templ` (source file)
@@ -360,7 +374,7 @@ import (
) )
func TestTagsFilter(t *testing.T) { func TestTagsFilter(t *testing.T) {
setup := setupTestServer(t) // ✅ Using test_helpers setup := setupDeviceTest(t) // ✅ Using test_helpers
defer setup.Server.Close() defer setup.Server.Close()
libraryID := setup.CreateLibrary(t, "Test Tags Library", "ebooks") libraryID := setup.CreateLibrary(t, "Test Tags Library", "ebooks")
@@ -392,12 +406,20 @@ func TestTagsFilter(t *testing.T) {
createBook("The Hobbit", "Tolkien", []string{"Fantasy", "Adventure"}) createBook("The Hobbit", "Tolkien", []string{"Fantasy", "Adventure"})
t.Run("Tags filter - Science Fiction", func(t *testing.T) { t.Run("Tags filter - Science Fiction", func(t *testing.T) {
req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&tags_filter=Science+Foundation", nil) req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&tags_filter=Science+Fiction", nil)
req.Header.Set("Authorization", "Bearer "+setup.UserToken) req.Header.Set("Authorization", "Bearer "+setup.UserToken)
// ... test implementation // ... test implementation
// Should return Foundation, Dune, Neuromancer (all have "Science Fiction" tag) // Should return Foundation, Dune, Neuromancer (all have "Science Fiction" tag)
}) })
t.Run("Tags filter - fuzzy match", func(t *testing.T) {
req := httptest.NewRequest("GET", "/api/media-items/search?library_id="+libraryID+"&tags_filter=Sci+Fi", nil)
req.Header.Set("Authorization", "Bearer "+setup.UserToken)
// ... test implementation
// Should return Foundation, Dune, Neuromancer (fuzzy matches "Science Fiction")
// Results should be ranked by similarity score
})
t.Run("Tags filter - Fantasy", func(t *testing.T) { t.Run("Tags filter - Fantasy", func(t *testing.T) {
// Should return only The Hobbit // Should return only The Hobbit
}) })
@@ -405,7 +427,7 @@ func TestTagsFilter(t *testing.T) {
t.Run("Tags autocomplete", func(t *testing.T) { t.Run("Tags autocomplete", func(t *testing.T) {
req := httptest.NewRequest("GET", "/api/media-items/search?tags=Sci&library_id="+libraryID, nil) req := httptest.NewRequest("GET", "/api/media-items/search?tags=Sci&library_id="+libraryID, nil)
req.Header.Set("Authorization", "Bearer "+setup.UserToken) req.Header.Set("Authorization", "Bearer "+setup.UserToken)
// Should return "Science Fiction" in autocomplete results // Should return "Science Fiction" in autocomplete results with fuzzy matching
}) })
t.Run("Backward compatibility - genre_filter still works", func(t *testing.T) { t.Run("Backward compatibility - genre_filter still works", func(t *testing.T) {
@@ -413,7 +435,7 @@ func TestTagsFilter(t *testing.T) {
}) })
t.Run("Combined filters - tags + author", func(t *testing.T) { t.Run("Combined filters - tags + author", func(t *testing.T) {
// tags_filter=Science+Fiction&author_filter=Asimov // tags_filter=Sci+Fi&author_filter=Asimov
// Should return only Foundation // Should return only Foundation
}) })
} }
@@ -422,6 +444,8 @@ func TestTagsFilter(t *testing.T) {
### Manual QA Checklist ### Manual QA Checklist
- [ ] Tags filter returns books with matching tags - [ ] Tags filter returns books with matching tags
- [ ] Tags filter uses fuzzy matching (e.g., "Sci Fi" matches "Science Fiction")
- [ ] Results are ranked by tag similarity score (best matches first)
- [ ] Tags autocomplete shows tag values as you type - [ ] Tags autocomplete shows tag values as you type
- [ ] Multiple tags in book are all searchable - [ ] Multiple tags in book are all searchable
- [ ] Combined filters work (tags + author + series) - [ ] Combined filters work (tags + author + series)
@@ -455,7 +479,7 @@ Search and filter media items with fuzzy matching and tag-based filtering.
| `library_id` | string | No | Filter by library UUID | | `library_id` | string | No | Filter by library UUID |
| `author_filter` | string | No | Fuzzy match author field | | `author_filter` | string | No | Fuzzy match author field |
| `series_filter` | string | No | Fuzzy match series field | | `series_filter` | string | No | Fuzzy match series field |
| `tags_filter` | string | No | **Exact match tags array** (NEW) | | `tags_filter` | string | No | **Fuzzy match tags array** (NEW) |
| `language_filter` | string | No | Fuzzy match language field | | `language_filter` | string | No | Fuzzy match language field |
| `year_min` | integer | No | Minimum copyright year | | `year_min` | integer | No | Minimum copyright year |
| `year_max` | integer | No | Maximum copyright year | | `year_max` | integer | No | Maximum copyright year |
@@ -468,21 +492,29 @@ Search and filter media items with fuzzy matching and tag-based filtering.
**New in v1.x:** The `tags_filter` parameter filters books by their tags array. **New in v1.x:** The `tags_filter` parameter filters books by their tags array.
- **Matching:** Exact match (not fuzzy) - the filter value must equal a tag - **Matching:** Fuzzy match using `word_similarity()` - matches tags with similarity > 0.3
- **Calibre Integration:** Works seamlessly with Calibre tags (which include genre) - **Calibre Integration:** Works seamlessly with Calibre tags (which include genre)
- **Result Ranking:** Best tag matches appear first in results
- **Example Tags:** `["Science Fiction", "Adventure", "Dystopian"]` - **Example Tags:** `["Science Fiction", "Adventure", "Dystopian"]`
**Examples:** **Fuzzy Matching Examples:**
```bash ```bash
# Filter by "Science Fiction" tag # Exact match: "Science Fiction"
GET /api/media-items/search?tags_filter=Science+Fiction GET /api/media-items/search?tags_filter=Science+Fiction
# Returns: All books with "Science Fiction" tag
# Fuzzy match: "Sci Fi" → "Science Fiction"
GET /api/media-items/search?tags_filter=Sci+Fi
# Returns: Books with "Science Fiction", "Sci-Fi", "Scifi" (score > 0.3)
# Combine tags with author # Combine tags with author
GET /api/media-items/search?tags_filter=Science+Fiction&author_filter=Asimov GET /api/media-items/search?tags_filter=Sci+Fi&author_filter=Asimov
# Returns: Science Fiction books by Asimov, best matches first
# Filter by multiple tags (returns books with ANY matching tag) # Fuzzy match with variations
GET /api/media-items/search?tags_filter=Cyberpunk GET /api/media-items/search?tags_filter=Cyber
# Returns: Books with "Cyberpunk", "Cyber", "Cyberthriller" tags
``` ```
### Backward Compatibility ### Backward Compatibility
@@ -519,7 +551,6 @@ tags_filter=Science+Fiction
} }
] ]
``` ```
```
### User Documentation ### User Documentation
@@ -560,8 +591,9 @@ You can combine tags with other filters:
- Tags from Calibre are automatically imported - Tags from Calibre are automatically imported
- You can add custom tags when editing book metadata - You can add custom tags when editing book metadata
- Tags are case-insensitive for searching - Tags use fuzzy matching - "Sci Fi" will match "Science Fiction"
- Autocomplete shows existing tags as you type - Best matching tags appear first in results
- Autocomplete shows existing tags as you type with relevance scores
``` ```
--- ---
@@ -570,9 +602,11 @@ You can combine tags with other filters:
### Commit 1: Backend SQL Layer ### Commit 1: Backend SQL Layer
``` ```
feat: add tags_filter to search query feat: add fuzzy tags_filter to search query
- Add tags_filter parameter to SearchMediaItemsUnified - Add fuzzy tags_filter parameter to SearchMediaItemsUnified
- Add EXISTS clause with word_similarity() for fuzzy tag matching
- Add tag similarity scoring to ORDER BY clause
- Add SearchTagsValues query for autocomplete - Add SearchTagsValues query for autocomplete
- Keep genre_filter for backward compatibility - Keep genre_filter for backward compatibility