Bookhoard/bookhoard
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 12 Wiki Activity

docs: update implementation guide with technical notes

Browse Source 
Update UNIFIED_SEARCH_IMPLEMENTATION.md with:

1. Technical note about sqlc v1.30.0 limitation:
   - CASE expressions in GROUP BY not supported
   - Solution: Use 4 separate simple queries instead of 1 complex query
   - Simpler approach that works correctly with current sqlc version

2. Implementation approach updates:
   - Service layer route to appropriate query based on field type
   - No changes needed to main.go or test helpers
   - SearchService created inside handler constructor

3. Phase 7 changes (skip):
   - No handler initialization changes needed
   - Follows FiltersService and CollectionService pattern
   - Rationale: more testable, simpler initialization

4. Updated timeline estimates
5. Updated success criteria

These notes clarify implementation decisions and provide context
for future maintainers.
This commit is contained in:
john-okeefe
2026-03-22 20:35:13 -04:00
parent 08435c8cd4
commit 057b595832
1 changed files with 168 additions and 94 deletions
Download Patch File Download Diff File Expand all files Collapse all files
UNIFIED_SEARCH_IMPLEMENTATION.md
+168 -94
View File
@@ -11,6 +11,9 @@
**Approach:** Surgical, incremental changes that reuse existing code, following PROJECT_GUIDELINES.md strictly.
**Technical Note - sqlc v1.30.0 Limitation:**
Autocomplete dropdowns use 4 separate simple queries (one per field type) instead of 1 complex query due to sqlc v1.30.0's inability to parse complex CASE expressions in GROUP BY clauses. This approach is simpler, works correctly with the current sqlc version, and the service layer routes to the appropriate query based on field type.
---
## Current State Analysis
@@ -186,44 +189,86 @@ ORDER BY
LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset');
```
**Add field value search query (for autocomplete dropdowns):**
**Add 4 separate field value search queries (for autocomplete dropdowns):**
**Note:** Using 4 separate simple queries instead of 1 complex query due to sqlc v1.30.0 limitation with CASE expressions in GROUP BY clauses. This approach is simpler and works correctly with the current sqlc version.
```sql
-- name: SearchFieldValues :many
SELECT DISTINCT
CASE sqlc.narg('field_type')
WHEN 'author' THEN mi.author
WHEN 'genre' THEN mi.genre
WHEN 'series' THEN mi.series
WHEN 'language' THEN mi.language
END as value,
-- name: SearchAuthorValues :many
SELECT
mi.author as value,
COUNT(*) as count,
CASE sqlc.narg('field_type')
WHEN 'author' THEN word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, ''))
WHEN 'genre' THEN word_similarity(sqlc.narg('search_query'), COALESCE(mi.genre, ''))
WHEN 'series' THEN word_similarity(sqlc.narg('search_query'), COALESCE(mi.series, ''))
WHEN 'language' THEN word_similarity(sqlc.narg('search_query'), COALESCE(mi.language, ''))
END as score
word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, ''))::float8 as score
FROM media_items mi
WHERE mi.library_id = sqlc.narg('library_id')
AND (
(sqlc.narg('field_type') = 'author' AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, '')) > 0.3) OR
(sqlc.narg('field_type') = 'genre' AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.genre, '')) > 0.3) OR
(sqlc.narg('field_type') = 'series' AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.series, '')) > 0.3) OR
(sqlc.narg('field_type') = 'language' AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.language, '')) > 0.3)
)
GROUP BY value, score
HAVING value IS NOT NULL AND value != ''
JOIN libraries l ON mi.library_id = l.id
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
WHERE COALESCE(lv.is_visible, true) = true
AND mi.library_id = sqlc.narg('library_id')
AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.author, '')) > 0.3
AND mi.author IS NOT NULL
AND mi.author != ''
GROUP BY mi.author
ORDER BY score DESC, count DESC
LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset');
-- name: SearchGenreValues :many
SELECT
mi.genre as value,
COUNT(*) as count,
word_similarity(sqlc.narg('search_query'), COALESCE(mi.genre, ''))::float8 as score
FROM media_items mi
JOIN libraries l ON mi.library_id = l.id
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
WHERE COALESCE(lv.is_visible, true) = true
AND mi.library_id = sqlc.narg('library_id')
AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.genre, '')) > 0.3
AND mi.genre IS NOT NULL
AND mi.genre != ''
GROUP BY mi.genre
ORDER BY score DESC, count DESC
LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset');
-- name: SearchSeriesValues :many
SELECT
mi.series as value,
COUNT(*) as count,
word_similarity(sqlc.narg('search_query'), COALESCE(mi.series, ''))::float8 as score
FROM media_items mi
JOIN libraries l ON mi.library_id = l.id
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
WHERE COALESCE(lv.is_visible, true) = true
AND mi.library_id = sqlc.narg('library_id')
AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.series, '')) > 0.3
AND mi.series IS NOT NULL
AND mi.series != ''
GROUP BY mi.series
ORDER BY score DESC, count DESC
LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset');
-- name: SearchLanguageValues :many
SELECT
mi.language as value,
COUNT(*) as count,
word_similarity(sqlc.narg('search_query'), COALESCE(mi.language, ''))::float8 as score
FROM media_items mi
JOIN libraries l ON mi.library_id = l.id
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
WHERE COALESCE(lv.is_visible, true) = true
AND mi.library_id = sqlc.narg('library_id')
AND word_similarity(sqlc.narg('search_query'), COALESCE(mi.language, '')) > 0.3
AND mi.language IS NOT NULL
AND mi.language != ''
GROUP BY mi.language
ORDER BY score DESC, count DESC
LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset');
```
**Regenerate Go code:**
```bash
go generate ./internal/database
sqlc generate
```
**Verify:** Check `internal/database/queries.sql.go` for new functions
**Verify:** Check `internal/database/queries.sql.go` for new functions: `SearchAuthorValues`, `SearchGenreValues`, `SearchSeriesValues`, `SearchLanguageValues`
---
@@ -343,43 +388,90 @@ type FieldValue struct {
// SearchFieldValues handles field-specific search for autocomplete dropdowns
// Returns distinct values with counts and similarity scores
// Uses 4 separate queries (one per field type) for sqlc v1.30.0 compatibility
func (s *SearchService) SearchFieldValues(ctx context.Context, params FieldSearchParams) ([]FieldValue, error) {
// Build database parameters
dbParams := database.SearchFieldValuesParams{
UserID: params.UserID,
LibraryID: params.LibraryID,
FieldType: pgtype.Text{String: params.FieldType, Valid: true},
SearchQuery: pgtype.Text{String: params.SearchQuery, Valid: true},
Limit: pgtype.Int4{Int32: params.Limit), Valid: true},
Offset: pgtype.Int4{Int32: params.Offset), Valid: true},
}
// Execute field values query
results, err := s.db.SearchFieldValues(ctx, dbParams)
if err != nil {
return nil, err
}
// Convert to service-level response type
fieldValues := make([]FieldValue, len(results))
for i, r := range results {
fieldValues[i] = FieldValue{
Value: r.Value,
Count: r.Count,
Score: r.Score,
switch params.FieldType {
case "author":
results, err := s.db.SearchAuthorValues(ctx, database.SearchAuthorValuesParams{
SearchQuery: params.SearchQuery,
UserID: params.UserID,
LibraryID: params.LibraryID,
Limit: int32(params.Limit),
Offset: int32(params.Offset),
})
if err != nil {
return nil, err
}
}
fieldValues := make([]FieldValue, len(results))
for i, r := range results {
fieldValues[i] = FieldValue{Value: r.Value, Count: r.Count, Score: r.Score}
}
return fieldValues, nil
return fieldValues, nil
case "genre":
results, err := s.db.SearchGenreValues(ctx, database.SearchGenreValuesParams{
SearchQuery: params.SearchQuery,
UserID: params.UserID,
LibraryID: params.LibraryID,
Limit: int32(params.Limit),
Offset: int32(params.Offset),
})
if err != nil {
return nil, err
}
fieldValues := make([]FieldValue, len(results))
for i, r := range results {
fieldValues[i] = FieldValue{Value: r.Value, Count: r.Count, Score: r.Score}
}
return fieldValues, nil
case "series":
results, err := s.db.SearchSeriesValues(ctx, database.SearchSeriesValuesParams{
SearchQuery: params.SearchQuery,
UserID: params.UserID,
LibraryID: params.LibraryID,
Limit: int32(params.Limit),
Offset: int32(params.Offset),
})
if err != nil {
return nil, err
}
fieldValues := make([]FieldValue, len(results))
for i, r := range results {
fieldValues[i] = FieldValue{Value: r.Value, Count: r.Count, Score: r.Score}
}
return fieldValues, nil
case "language":
results, err := s.db.SearchLanguageValues(ctx, database.SearchLanguageValuesParams{
SearchQuery: params.SearchQuery,
UserID: params.UserID,
LibraryID: params.LibraryID,
Limit: int32(params.Limit),
Offset: int32(params.Offset),
})
if err != nil {
return nil, err
}
fieldValues := make([]FieldValue, len(results))
for i, r := range results {
fieldValues[i] = FieldValue{Value: r.Value, Count: r.Count, Score: r.Score}
}
return fieldValues, nil
default:
return []FieldValue{}, nil
}
}
```
**Verification:**
```bash
go build ./internal/handlers
```
**Note:** SearchService is created inside the constructor (not in main.go), following the pattern of `FiltersService` and `CollectionService` which are created inside their respective handlers.
**Note:** Router initialization will also need updating (see Phase 7)
---
### Phase 4: Update Handler to Use Search Service
**File:** `internal/handlers/media.go`
#### 4.1 Fix Search Box Endpoint
@@ -913,30 +1005,19 @@ params:
---
### Phase 7: Update Handler Initialization
### Phase 7: No Changes Needed (Skip)
**Files to update:**
- `cmd/server/main.go` (line ~122-126)
- `cmd/server/tests/test_helpers_test.go` (line ~468-472)
**No changes required to:**
- `cmd/server/main.go`
- `cmd/server/tests/test_helpers_test.go`
**In both files, find the MediaHandler initialization:**
SearchService follows the pattern of `FiltersService` and `CollectionService` - it's created **inside** the handler constructor, not in main.go. This keeps handler dependencies self-contained.
```go
// BEFORE
mediaHandler := handlers.NewMediaHandler(queries, libraryService, worker)
// AFTER
searchService := services.NewSearchService(queries)
mediaHandler := handlers.NewMediaHandler(queries, libraryService, searchService, worker)
```
**Note:** This follows the same pattern as `conversionService` which is created in main.go (line 118) and passed to handlers.
**Verify compilation:**
```bash
go build ./cmd/server
go test ./cmd/server/tests -run TestNonExistent # Compile test only
```
**Rationale:**
- Matches established pattern (`FiltersService`, `CollectionService`)
- Handler owns its service dependencies
- Simpler initialization - no handler-specific services in main.go
- More testable
---
@@ -1080,22 +1161,14 @@ git add internal/handlers/media.go
git commit -m "refactor: update SearchMediaItems handler to use SearchService
- Add searchService to MediaHandler struct
- Update NewMediaHandler constructor
- Create SearchService inside NewMediaHandler constructor
- Refactor SearchMediaItems to delegate to service layer
- Add handleUnifiedSearch method (thin wrapper)
- Add handleFieldValuesSearch method (thin wrapper)
- Handler now only extracts params and calls service"
- Handler now only extracts params and calls service
- Follows pattern of FiltersService and CollectionService"
# Commit 6: Update handler initialization
git add cmd/server/main.go cmd/server/tests/test_helpers_test.go
git commit -m "refactor: add SearchService to handler initialization
- Instantiate SearchService in main.go and test_helpers_test.go
- Pass searchService to MediaHandler constructor
- Follow existing pattern (like conversionService)
- Update both production and test initialization"
# Commit 7: Frontend templates
# Commit 6: Frontend templates
git add templates/bookshelf.templ
git commit -m "feat: fix search box to use /search endpoint with autocomplete
@@ -1122,7 +1195,7 @@ git commit -m "test: add comprehensive tests for unified search endpoint
- Test combined search + filters
- Test field-specific search for dropdowns
- Test year range and boolean filters
- Use setupTestServer helper following PROJECT_GUIDELINES.md"
- Use setupDeviceTest helper following PROJECT_GUIDELINES.md"
# Commit 10: Documentation
git add docs/developer/api/media-items/search_media_items.md
@@ -1197,8 +1270,8 @@ git show HEAD~1:templates/bookshelf.templ > templates/bookshelf.templ
- Phase 1 (Database): 30 minutes
- Phase 2 (SQL Queries): 1 hour
- Phase 3 (Search Service): 2 hours
- Phase 4 (Handler Updates): 1 hour
- Phase 5 (Router Updates): 15 minutes
- Phase 4 (Handler Update): 1 hour
- Phase 5 (No Changes): 0 minutes
- Phase 6 (Frontend): 2 hours
- Phase 7 (TypeScript): 1 hour
- Phase 8 (Tests): 2 hours
@@ -1206,13 +1279,14 @@ git show HEAD~1:templates/bookshelf.templ > templates/bookshelf.templ
- Phase 10 (Cleanup): 30 minutes
- Phase 11 (Verification): 1 hour
**Total: ~12 hours**
**Total: ~11.5 hours**
---
## Success Criteria
✅ All business logic in `services/search.go` (service layer pattern)
✅ SearchService created inside NewMediaHandler constructor (not main.go)
✅ Handler is thin - only extracts params and calls service
✅ All text filters use fuzzy matching (pg_trgm)
✅ Exact match with quotes works
@@ -1220,7 +1294,7 @@ git show HEAD~1:templates/bookshelf.templ > templates/bookshelf.templ
✅ Autocomplete dropdowns populate correctly
✅ Years/booleans remain exact match
✅ Saved filters load correctly
✅ All tests pass using `setupTestServer` helper
✅ All tests pass using `setupDeviceTest` helper
✅ Documentation updated
✅ No breaking changes to saved filters
✅ Deprecated `/filtered` endpoint removed