Add comprehensive implementation plan for full-text documentation search using backend API endpoint instead of build-time Lunr index. Changes: - Create DOCS_SEARCH_IMPLEMENTATION.md with complete implementation guide - Backend: internal/docs/search.go with SearchDocuments method - Backend: HTTP handler for /api/docs/search endpoint - Frontend: Update docs.ts to use API instead of client-side Lunr - Testing: Unit tests (search_test.go) and integration tests (docs_search_test.go) - Bruno: Add Search Docs.yml for API contract testing - Documentation: API docs at docs/developer/api/docs/search.md Key features: - Full-text search across 152 markdown documentation files - Case-insensitive matching with snippet extraction - RESTful API endpoint (no build step required) - Follows PROJECT_GUIDELINES.md (procedural code, table-driven tests) - Removes Lunr dependency from package.json - Consistent with existing /api/media-items/search pattern Estimated effort: 3-4 hours Testing strategy: Unit tests + integration tests + Bruno YAML
1103 lines
26 KiB
Markdown
1103 lines
26 KiB
Markdown
# Implementation Plan: Docs Search API
|
|
|
|
## Overview
|
|
Add backend full-text search for 152 documentation files using a new `/api/docs/search` endpoint, replacing the planned build-time Lunr approach with runtime server-side search.
|
|
|
|
**Type**: Full-stack task (backend API + frontend changes)
|
|
**Status**: Planned
|
|
**Priority**: Medium
|
|
**Estimated effort**: 3-4 hours
|
|
|
|
---
|
|
|
|
## Table of Contents
|
|
1. [Architecture Decision](#architecture-decision)
|
|
2. [Backend Implementation](#backend-implementation)
|
|
3. [Frontend Changes](#frontend-changes)
|
|
4. [Testing Strategy](#testing-strategy)
|
|
5. [Dependencies to Remove](#dependencies-to-remove)
|
|
6. [Documentation Updates](#documentation-updates)
|
|
7. [Implementation Checklist](#implementation-checklist)
|
|
|
|
---
|
|
|
|
## Architecture Decision
|
|
|
|
### Why Backend API Instead of Build-time Lunr?
|
|
|
|
| Aspect | Build-time Lunr (Original Plan) | Backend API (This Plan) |
|
|
|--------|----------------------------------|-------------------------|
|
|
| Build step | ✅ Required | ❌ None |
|
|
| Network request | ❌ Client-side only | ✅ API call |
|
|
| Server load | ❌ None | ✅ Minimal (152 docs) |
|
|
| Offline support | ✅ Yes | ❌ No |
|
|
| Search speed | ⚡ Instant | 🌐 Network latency |
|
|
| Bundle size | +50KB | No change |
|
|
| Always current | ❌ Need rebuild | ✅ Yes |
|
|
| Consistency | New pattern | Matches existing `/api/media-items/search` |
|
|
|
|
**Decision**: Backend API is simpler and more consistent with existing architecture.
|
|
|
|
---
|
|
|
|
## Backend Implementation
|
|
|
|
### 1. Add Search Method to DocsHandler
|
|
|
|
**File**: `internal/docs/search.go` (new file)
|
|
|
|
```go
|
|
package docs
|
|
|
|
import (
|
|
"bufio"
|
|
"fmt"
|
|
"io/fs"
|
|
"path/filepath"
|
|
"strings"
|
|
"unicode"
|
|
)
|
|
|
|
// SearchResult represents a single search result
|
|
type SearchResult struct {
|
|
Path string `json:"path"`
|
|
Title string `json:"title"`
|
|
Section string `json:"section"`
|
|
Snippet string `json:"snippet,omitempty"`
|
|
}
|
|
|
|
// SearchDocuments performs full-text search across all markdown files
|
|
func (h *DocsHandler) SearchDocuments(query string) ([]SearchResult, error) {
|
|
// Validate query
|
|
query = strings.TrimSpace(query)
|
|
if len(query) < 2 {
|
|
return []SearchResult{}, nil
|
|
}
|
|
|
|
// Convert to lowercase for case-insensitive search
|
|
queryLower := strings.ToLower(query)
|
|
|
|
var results []SearchResult
|
|
|
|
// Walk through all markdown files
|
|
err := fs.WalkDir(h.docsFS, ".", func(path string, d fs.DirEntry, err error) error {
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
// Skip directories and non-markdown files
|
|
if d.IsDir() || !strings.HasSuffix(path, ".md") {
|
|
return nil
|
|
}
|
|
|
|
// Read file content
|
|
content, err := fs.ReadFile(h.docsFS, path)
|
|
if err != nil {
|
|
// Log but continue - don't fail entire search
|
|
fmt.Printf("Warning: failed to read %s: %v\n", path, err)
|
|
return nil
|
|
}
|
|
|
|
// Extract title and check for matches
|
|
title := h.extractTitle(content)
|
|
section := h.getSectionFromPath(path)
|
|
textContent := h.markdownToText(content)
|
|
|
|
// Check if query matches
|
|
if h.matchesQuery(textContent, queryLower) {
|
|
result := SearchResult{
|
|
Path: path,
|
|
Title: title,
|
|
Section: section,
|
|
}
|
|
|
|
// Generate snippet with highlighted match
|
|
snippet := h.extractSnippet(textContent, queryLower)
|
|
if snippet != "" {
|
|
result.Snippet = snippet
|
|
}
|
|
|
|
results = append(results, result)
|
|
}
|
|
|
|
return nil
|
|
})
|
|
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to walk docs directory: %w", err)
|
|
}
|
|
|
|
return results, nil
|
|
}
|
|
|
|
// getSectionFromPath derives section name from file path
|
|
func (h *DocsHandler) getSectionFromPath(path string) string {
|
|
// Remove .md extension
|
|
path = strings.TrimSuffix(path, ".md")
|
|
|
|
// Split by directory separators
|
|
parts := strings.Split(path, string(filepath.Separator))
|
|
|
|
// Filter out empty strings and index.md
|
|
var filtered []string
|
|
for _, part := range parts {
|
|
if part != "" && part != "index.md" && part != "index" {
|
|
// Convert kebab-case to Title Case
|
|
filtered = append(filtered, kebabToTitle(part))
|
|
}
|
|
}
|
|
|
|
// Join with " > "
|
|
return strings.Join(filtered, " > ")
|
|
}
|
|
|
|
// kebabToTitle converts kebab-case to Title Case
|
|
func kebabToTitle(s string) string {
|
|
words := strings.Split(s, "-")
|
|
for i, word := range words {
|
|
if len(word) > 0 {
|
|
// Capitalize first letter
|
|
runes := []rune(word)
|
|
runes[0] = unicode.ToUpper(runes[0])
|
|
words[i] = string(runes)
|
|
}
|
|
}
|
|
return strings.Join(words, " ")
|
|
}
|
|
|
|
// markdownToText converts markdown content to plain text for searching
|
|
func (h *DocsHandler) markdownToText(content []byte) string {
|
|
// Remove markdown syntax but keep text
|
|
lines := strings.Split(string(content), "\n")
|
|
var textLines []string
|
|
|
|
for _, line := range lines {
|
|
line = strings.TrimSpace(line)
|
|
|
|
// Skip code blocks
|
|
if strings.HasPrefix(line, "```") {
|
|
continue
|
|
}
|
|
|
|
// Remove markdown headers
|
|
line = strings.TrimLeft(line, "#")
|
|
|
|
// Remove bold/italic markers
|
|
line = strings.ReplaceAll(line, "**", "")
|
|
line = strings.ReplaceAll(line, "*", "")
|
|
line = strings.ReplaceAll(line, "__", "")
|
|
line = strings.ReplaceAll(line, "_", "")
|
|
|
|
// Remove links but keep text
|
|
line = h.removeMarkdownLinks(line)
|
|
|
|
// Remove code inline
|
|
line = strings.ReplaceAll(line, "`", "")
|
|
|
|
if line != "" {
|
|
textLines = append(textLines, line)
|
|
}
|
|
}
|
|
|
|
return strings.Join(textLines, " ")
|
|
}
|
|
|
|
// removeMarkdownLinks removes markdown link syntax
|
|
func (h *DocsHandler) removeMarkdownLinks(line string) string {
|
|
// Simple approach: remove [text](url) patterns
|
|
result := line
|
|
for {
|
|
start := strings.Index(result, "[")
|
|
if start == -1 {
|
|
break
|
|
}
|
|
end := strings.Index(result[start:], "](")
|
|
if end == -1 {
|
|
break
|
|
}
|
|
end += start + 2 // include "]("
|
|
|
|
// Find closing paren
|
|
closeParen := strings.Index(result[end:], ")")
|
|
if closeParen == -1 {
|
|
break
|
|
}
|
|
closeParen += end
|
|
|
|
// Extract link text
|
|
linkText := result[start+1 : start+end-start-2]
|
|
|
|
// Replace entire link with just text
|
|
result = result[:start] + linkText + result[closeParen+1:]
|
|
}
|
|
return result
|
|
}
|
|
|
|
// matchesQuery checks if content matches the search query
|
|
func (h *DocsHandler) matchesQuery(content, queryLower string) bool {
|
|
contentLower := strings.ToLower(content)
|
|
return strings.Contains(contentLower, queryLower)
|
|
}
|
|
|
|
// extractSnippet extracts a snippet around the matched term
|
|
func (h *DocsHandler) extractSnippet(content, queryLower string) string {
|
|
contentLower := strings.ToLower(content)
|
|
|
|
// Find first match
|
|
idx := strings.Index(contentLower, queryLower)
|
|
if idx == -1 {
|
|
return ""
|
|
}
|
|
|
|
// Extract context around match (100 chars before and after)
|
|
start := idx - 50
|
|
if start < 0 {
|
|
start = 0
|
|
}
|
|
end := idx + len(queryLower) + 50
|
|
if end > len(content) {
|
|
end = len(content)
|
|
}
|
|
|
|
snippet := content[start:end]
|
|
|
|
// Add ellipsis if truncated
|
|
if start > 0 {
|
|
snippet = "..." + snippet
|
|
}
|
|
if end < len(content) {
|
|
snippet = snippet + "..."
|
|
}
|
|
|
|
return snippet
|
|
}
|
|
```
|
|
|
|
### 2. Add HTTP Handler
|
|
|
|
**File**: `internal/docs/http_handler.go` (existing file - append to it)
|
|
|
|
```go
|
|
package docs
|
|
|
|
import (
|
|
"encoding/json"
|
|
"net/http"
|
|
"strings"
|
|
|
|
"github.com/go-chi/chi/v5"
|
|
)
|
|
|
|
// HandleSearch handles documentation search requests
|
|
func (h *DocsHandler) HandleSearch(w http.ResponseWriter, r *http.Request) {
|
|
// Extract query parameter
|
|
query := r.URL.Query().Get("q")
|
|
query = strings.TrimSpace(query)
|
|
|
|
// Validate query
|
|
if query == "" {
|
|
w.WriteHeader(http.StatusBadRequest)
|
|
json.NewEncoder(w).Encode(map[string]interface{}{
|
|
"error": "query parameter 'q' is required",
|
|
})
|
|
return
|
|
}
|
|
|
|
if len(query) < 2 {
|
|
w.WriteHeader(http.StatusBadRequest)
|
|
json.NewEncoder(w).Encode(map[string]interface{}{
|
|
"error": "query must be at least 2 characters",
|
|
})
|
|
return
|
|
}
|
|
|
|
// Perform search
|
|
results, err := h.SearchDocuments(query)
|
|
if err != nil {
|
|
w.WriteHeader(http.StatusInternalServerError)
|
|
json.NewEncoder(w).Encode(map[string]interface{}{
|
|
"error": "failed to search documents",
|
|
})
|
|
return
|
|
}
|
|
|
|
// Return results
|
|
w.Header().Set("Content-Type", "application/json")
|
|
json.NewEncoder(w).Encode(map[string]interface{}{
|
|
"query": query,
|
|
"count": len(results),
|
|
"results": results,
|
|
})
|
|
}
|
|
|
|
// RegisterDocsSearchRoutes registers search endpoints
|
|
func RegisterDocsSearchRoutes(r chi.Router, handler *DocsHandler) {
|
|
r.Get("/api/docs/search", handler.HandleSearch)
|
|
}
|
|
```
|
|
|
|
### 3. Register Route
|
|
|
|
**File**: `internal/router/docs.go` (modify existing)
|
|
|
|
Add the search route registration:
|
|
|
|
```go
|
|
func RegisterDocsRoutes(r chi.Router, docsPath string) {
|
|
handler := docs.NewDocsHandler(docsPath)
|
|
|
|
// Existing routes...
|
|
r.Get("/docs", docsHandler)
|
|
r.Get("/docs/*", docsHandler)
|
|
|
|
// NEW: Search route
|
|
docs.RegisterDocsSearchRoutes(r, handler)
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Frontend Changes
|
|
|
|
### 1. Update docs.ts to Remove Lunr
|
|
|
|
**File**: `web/src/docs.ts`
|
|
|
|
**Remove broken imports** (lines 1-4):
|
|
```typescript
|
|
// DELETE THESE LINES:
|
|
import { docs } from "../data/docs.json";
|
|
import searchIndex from "../data/search_index.json";
|
|
import * as lunr from "lunr";
|
|
import hljs from "highlight.js";
|
|
```
|
|
|
|
**Replace search function** (lines 46-88):
|
|
|
|
```typescript
|
|
// BEFORE:
|
|
function performDocsSearch(query: string): void {
|
|
const searchResults = document.getElementById("docs-search-results");
|
|
if (!searchResults) return;
|
|
|
|
try {
|
|
const idx = lunr.Builder.loadJs(searchIndex);
|
|
// ... lunr search logic
|
|
} catch (error) {
|
|
console.error("Search error:", error);
|
|
}
|
|
}
|
|
|
|
// AFTER:
|
|
async function performDocsSearch(query: string): Promise<void> {
|
|
const searchResults = document.getElementById("docs-search-results");
|
|
if (!searchResults) return;
|
|
|
|
try {
|
|
// Show loading state
|
|
searchResults.innerHTML = '<p class="p-2 text-sm" style="color: var(--text-secondary)">Searching...</p>';
|
|
searchResults.classList.remove("hidden");
|
|
|
|
// Fetch from backend API
|
|
const response = await fetch(`/api/docs/search?q=${encodeURIComponent(query)}`);
|
|
|
|
if (!response.ok) {
|
|
throw new Error(`Search failed: ${response.statusText}`);
|
|
}
|
|
|
|
const data = await response.json();
|
|
|
|
if (!data.results || data.results.length === 0) {
|
|
searchResults.innerHTML =
|
|
'<p class="p-2 text-sm" style="color: var(--text-secondary)">No results found</p>';
|
|
} else {
|
|
searchResults.innerHTML = data.results
|
|
.slice(0, 10)
|
|
.map((result: { path: string; title: string; section: string; snippet?: string }) => {
|
|
return `
|
|
<a href="/docs/${result.path}" class="block p-2 hover:bg-opacity-50 transition-colors" style="background-color: var(--bg-secondary)">
|
|
<p class="font-medium text-sm" style="color: var(--text-primary)">${result.title || result.path}</p>
|
|
${result.section ? `<p class="text-xs" style="color: var(--text-secondary)">${result.section}</p>` : ""}
|
|
${result.snippet ? `<p class="text-xs mt-1" style="color: var(--text-secondary)">${result.snippet}</p>` : ""}
|
|
</a>
|
|
`;
|
|
})
|
|
.join("");
|
|
}
|
|
|
|
searchResults.classList.remove("hidden");
|
|
} catch (error) {
|
|
console.error("Search error:", error);
|
|
searchResults.innerHTML =
|
|
'<p class="p-2 text-sm" style="color: var(--text-secondary)">Search error</p>';
|
|
searchResults.classList.remove("hidden");
|
|
}
|
|
}
|
|
```
|
|
|
|
**Keep the rest** - toggleSidebar, initializeDocsSearch, and Alpine registration remain the same.
|
|
|
|
### 2. Remove Lunr Dependency
|
|
|
|
**File**: `package.json`
|
|
|
|
**Remove from dependencies** (line 16):
|
|
```json
|
|
"lunr": "^2.3.9", // DELETE THIS LINE
|
|
```
|
|
|
|
**Run**:
|
|
```bash
|
|
npm uninstall lunr
|
|
```
|
|
|
|
---
|
|
|
|
## Testing Strategy
|
|
|
|
### 1. Unit Tests (No API Required)
|
|
|
|
**File**: `internal/docs/search_test.go` (new file)
|
|
|
|
```go
|
|
package docs
|
|
|
|
import (
|
|
"testing"
|
|
|
|
"github.com/stretchr/testify/assert"
|
|
"github.com/stretchr/testify/require"
|
|
)
|
|
|
|
func TestMarkdownToText(t *testing.T) {
|
|
handler := &DocsHandler{}
|
|
|
|
testCases := []struct {
|
|
name string
|
|
input string
|
|
expected string
|
|
}{
|
|
{
|
|
name: "Simple text",
|
|
input: "Hello world",
|
|
expected: "Hello world",
|
|
},
|
|
{
|
|
name: "Remove headers",
|
|
input: "# Title\n\nSome content",
|
|
expected: "Title Some content",
|
|
},
|
|
{
|
|
name: "Remove bold",
|
|
input: "This is **bold** text",
|
|
expected: "This is bold text",
|
|
},
|
|
{
|
|
name: "Remove links",
|
|
input: "See [the docs](/docs/index.md) for more",
|
|
expected: "See the docs for more",
|
|
},
|
|
{
|
|
name: "Remove code blocks",
|
|
input: "```go\nfunc test() {}\n```",
|
|
expected: "",
|
|
},
|
|
}
|
|
|
|
for _, tc := range testCases {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
result := handler.markdownToText([]byte(tc.input))
|
|
assert.Equal(t, tc.expected, result)
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestGetSectionFromPath(t *testing.T) {
|
|
handler := &DocsHandler{}
|
|
|
|
testCases := []struct {
|
|
name string
|
|
path string
|
|
expected string
|
|
}{
|
|
{
|
|
name: "API endpoint",
|
|
path: "developer/api/auth/login.md",
|
|
expected: "Developer > Api > Auth > Login",
|
|
},
|
|
{
|
|
name: "User guide",
|
|
path: "user/devices/kobo-setup.md",
|
|
expected: "User > Devices > Kobo Setup",
|
|
},
|
|
{
|
|
name: "Index file",
|
|
path: "index.md",
|
|
expected: "",
|
|
},
|
|
}
|
|
|
|
for _, tc := range testCases {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
result := handler.getSectionFromPath(tc.path)
|
|
assert.Equal(t, tc.expected, result)
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestMatchesQuery(t *testing.T) {
|
|
handler := &DocsHandler{}
|
|
|
|
testCases := []struct {
|
|
name string
|
|
content string
|
|
query string
|
|
expected bool
|
|
}{
|
|
{
|
|
name: "Exact match",
|
|
content: "Kobo setup guide for e-readers",
|
|
query: "kobo",
|
|
expected: true,
|
|
},
|
|
{
|
|
name: "Case insensitive",
|
|
content: "Kobo Setup Guide",
|
|
query: "KOBO",
|
|
expected: true,
|
|
},
|
|
{
|
|
name: "No match",
|
|
content: "iPhone setup guide",
|
|
query: "kobo",
|
|
expected: false,
|
|
},
|
|
{
|
|
name: "Partial match",
|
|
content: "Configure your Kobo device",
|
|
query: "kobo device",
|
|
expected: true,
|
|
},
|
|
}
|
|
|
|
for _, tc := range testCases {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
result := handler.matchesQuery(tc.content, tc.query)
|
|
assert.Equal(t, tc.expected, result)
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestExtractSnippet(t *testing.T) {
|
|
handler := &DocsHandler{}
|
|
|
|
testCases := []struct {
|
|
name string
|
|
content string
|
|
query string
|
|
expectSnippet bool
|
|
}{
|
|
{
|
|
name: "Match in middle",
|
|
content: "This is some long text with Kobo device configuration options",
|
|
query: "kobo device",
|
|
expectSnippet: true,
|
|
},
|
|
{
|
|
name: "No match",
|
|
content: "This is some text without the term",
|
|
query: "kobo",
|
|
expectSnippet: false,
|
|
},
|
|
{
|
|
name: "Match at start",
|
|
content: "Kobo setup guide for e-readers with detailed instructions",
|
|
query: "kobo",
|
|
expectSnippet: true,
|
|
},
|
|
}
|
|
|
|
for _, tc := range testCases {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
result := handler.extractSnippet(tc.content, tc.query)
|
|
if tc.expectSnippet {
|
|
assert.NotEmpty(t, result)
|
|
assert.Contains(t, result, "...")
|
|
} else {
|
|
assert.Empty(t, result)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestKebabToTitle(t *testing.T) {
|
|
testCases := []struct {
|
|
input string
|
|
expected string
|
|
}{
|
|
{"kobo-setup", "Kobo Setup"},
|
|
{"api-reference", "Api Reference"},
|
|
{"user-guide", "User Guide"},
|
|
{"auth", "Auth"},
|
|
}
|
|
|
|
for _, tc := range testCases {
|
|
t.Run(tc.input, func(t *testing.T) {
|
|
result := kebabToTitle(tc.input)
|
|
assert.Equal(t, tc.expected, result)
|
|
})
|
|
}
|
|
}
|
|
```
|
|
|
|
### 2. Integration Tests (with API)
|
|
|
|
**File**: `cmd/server/tests/docs_search_test.go` (new file)
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"encoding/json"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"testing"
|
|
|
|
"github.com/stretchr/testify/assert"
|
|
"github.com/stretchr/testify/require"
|
|
)
|
|
|
|
func TestDocsSearchAPI(t *testing.T) {
|
|
// Setup test server
|
|
ts := setupTestServer(t)
|
|
defer ts.cleanup(t)
|
|
|
|
testCases := []struct {
|
|
name string
|
|
query string
|
|
expectedStatus int
|
|
expectResults bool
|
|
description string
|
|
}{
|
|
{
|
|
name: "Empty query",
|
|
query: "",
|
|
expectedStatus: http.StatusBadRequest,
|
|
expectResults: false,
|
|
description: "Should reject empty query",
|
|
},
|
|
{
|
|
name: "Single character",
|
|
query: "a",
|
|
expectedStatus: http.StatusBadRequest,
|
|
expectResults: false,
|
|
description: "Should reject queries < 2 chars",
|
|
},
|
|
{
|
|
name: "Valid search - kobo",
|
|
query: "kobo",
|
|
expectedStatus: http.StatusOK,
|
|
expectResults: true,
|
|
description: "Should find Kobo documentation",
|
|
},
|
|
{
|
|
name: "Valid search - API",
|
|
query: "API",
|
|
expectedStatus: http.StatusOK,
|
|
expectResults: true,
|
|
description: "Should find API documentation (case insensitive)",
|
|
},
|
|
{
|
|
name: "Valid search - no results",
|
|
query: "xyznonexistent",
|
|
expectedStatus: http.StatusOK,
|
|
expectResults: false,
|
|
description: "Should return empty results for non-existent term",
|
|
},
|
|
{
|
|
name: "Valid search - sync",
|
|
query: "sync",
|
|
expectedStatus: http.StatusOK,
|
|
expectResults: true,
|
|
description: "Should find sync documentation",
|
|
},
|
|
}
|
|
|
|
for _, tc := range testCases {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
// Create request
|
|
req := httptest.NewRequest("GET", "/api/docs/search?q="+tc.query, nil)
|
|
rr := httptest.NewRecorder()
|
|
|
|
// Serve request
|
|
ts.handler.ServeHTTP(rr, req)
|
|
|
|
// Check status
|
|
assert.Equal(t, tc.expectedStatus, rr.Code, tc.description)
|
|
|
|
// Parse response
|
|
var response map[string]interface{}
|
|
err := json.Unmarshal(rr.Body.Bytes(), &response)
|
|
require.NoError(t, err)
|
|
|
|
// Validate response structure
|
|
if tc.expectedStatus == http.StatusOK {
|
|
assert.Contains(t, response, "query")
|
|
assert.Contains(t, response, "count")
|
|
assert.Contains(t, response, "results")
|
|
|
|
results, ok := response["results"].([]interface{})
|
|
require.True(t, ok, "results should be an array")
|
|
|
|
if tc.expectResults {
|
|
assert.Greater(t, len(results), 0, "Should have results")
|
|
|
|
// Check first result structure
|
|
if len(results) > 0 {
|
|
firstResult, ok := results[0].(map[string]interface{})
|
|
require.True(t, ok, "result should be an object")
|
|
|
|
assert.Contains(t, firstResult, "path")
|
|
assert.Contains(t, firstResult, "title")
|
|
assert.Contains(t, firstResult, "section")
|
|
}
|
|
} else {
|
|
assert.Equal(t, 0, len(results), "Should have no results")
|
|
}
|
|
} else {
|
|
assert.Contains(t, response, "error")
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestDocsSearchResultStructure(t *testing.T) {
|
|
ts := setupTestServer(t)
|
|
defer ts.cleanup(t)
|
|
|
|
// Search for a known term
|
|
req := httptest.NewRequest("GET", "/api/docs/search?q=kobo", nil)
|
|
rr := httptest.NewRecorder()
|
|
|
|
ts.handler.ServeHTTP(rr, req)
|
|
|
|
assert.Equal(t, http.StatusOK, rr.Code)
|
|
|
|
var response map[string]interface{}
|
|
err := json.Unmarshal(rr.Body.Bytes(), &response)
|
|
require.NoError(t, err)
|
|
|
|
results, ok := response["results"].([]interface{})
|
|
require.True(t, ok)
|
|
require.Greater(t, len(results), 0, "Should have at least one result for 'kobo'")
|
|
|
|
// Validate first result has all required fields
|
|
firstResult := results[0].(map[string]interface{})
|
|
|
|
assert.Contains(t, firstResult, "path", "Should have path")
|
|
assert.Contains(t, firstResult, "title", "Should have title")
|
|
assert.Contains(t, firstResult, "section", "Should have section")
|
|
|
|
// Check types
|
|
assert.IsType(t, "", firstResult["path"])
|
|
assert.IsType(t, "", firstResult["title"])
|
|
assert.IsType(t, "", firstResult["section"])
|
|
|
|
// Snippet is optional
|
|
if snippet, ok := firstResult["snippet"]; ok {
|
|
assert.IsType(t, "", snippet)
|
|
}
|
|
}
|
|
```
|
|
|
|
**Note**: `setupTestServer()` helper should be called once at the test function level (not per subtest). If test_helpers.go doesn't exist yet, create it following existing test patterns in `cmd/server/tests/`.
|
|
|
|
### 3. Bruno OpenCollection YAML
|
|
|
|
**File**: `bruno/docs/Search Docs.yml` (new file)
|
|
|
|
```yaml
|
|
info:
|
|
name: Search Docs
|
|
type: http
|
|
seq: 1
|
|
|
|
http:
|
|
method: GET
|
|
url: '{{base_url}}/api/docs/search?q=kobo'
|
|
auth: none
|
|
|
|
docs: |-
|
|
## Search Documentation
|
|
|
|
Performs full-text search across all documentation files.
|
|
|
|
**Method:** GET
|
|
|
|
**Endpoint:** /api/docs/search
|
|
|
|
**Query Parameters:**
|
|
- `q` (string, required): Search query (minimum 2 characters)
|
|
|
|
**Authentication:** Not required (public documentation)
|
|
|
|
**Response:**
|
|
```json
|
|
{
|
|
"query": "kobo",
|
|
"count": 5,
|
|
"results": [
|
|
{
|
|
"path": "user/devices/kobo-setup.md",
|
|
"title": "Kobo Setup Guide",
|
|
"section": "User > Devices > Kobo Setup",
|
|
"snippet": "Complete Kobo e-reader configuration instructions..."
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
**Status Codes:**
|
|
- 200: Success (results may be empty)
|
|
- 400: Bad request (missing or invalid query)
|
|
- 500: Internal server error
|
|
|
|
**Examples:**
|
|
```bash
|
|
# Search for "kobo"
|
|
GET /api/docs/search?q=kobo
|
|
|
|
# Search for "API authentication"
|
|
GET /api/docs/search?q=api%20authentication
|
|
|
|
# Case insensitive
|
|
GET /api/docs/search?q=KOBO
|
|
|
|
# Minimum 2 characters
|
|
GET /api/docs/search?q=a # Returns 400 Bad Request
|
|
```
|
|
|
|
---
|
|
|
|
## Dependencies to Remove
|
|
|
|
### 1. Lunr (npm package)
|
|
|
|
**File**: `package.json`
|
|
|
|
**Before** (line 16):
|
|
```json
|
|
"lunr": "^2.3.9",
|
|
```
|
|
|
|
**After**: Delete this line
|
|
|
|
**Run**:
|
|
```bash
|
|
npm uninstall lunr
|
|
```
|
|
|
|
### 2. Clean Up docs.ts
|
|
|
|
**File**: `web/src/docs.ts`
|
|
|
|
**Remove imports** (lines 1-4):
|
|
```typescript
|
|
import { docs } from "../data/docs.json";
|
|
import searchIndex from "../data/search_index.json";
|
|
import * as lunr from "lunr";
|
|
import hljs from "highlight.js";
|
|
```
|
|
|
|
**Note**: `hljs` import can also be removed if not used elsewhere in the file.
|
|
|
|
---
|
|
|
|
## Documentation Updates
|
|
|
|
### 1. API Documentation
|
|
|
|
**File**: `docs/developer/api/docs/search.md` (new file)
|
|
|
|
```markdown
|
|
# Search Documentation
|
|
|
|
Search across all documentation files with full-text search.
|
|
|
|
## Endpoint
|
|
|
|
`GET /api/docs/search`
|
|
|
|
## Authentication
|
|
|
|
Not required (public documentation).
|
|
|
|
## Query Parameters
|
|
|
|
| Parameter | Type | Required | Description |
|
|
|-----------|--------|----------|----------------------------------|
|
|
| q | string | Yes | Search query (min 2 characters) |
|
|
|
|
## Response
|
|
|
|
### Success (200 OK)
|
|
|
|
```json
|
|
{
|
|
"query": "kobo",
|
|
"count": 5,
|
|
"results": [
|
|
{
|
|
"path": "user/devices/kobo-setup.md",
|
|
"title": "Kobo Setup Guide",
|
|
"section": "User > Devices > Kobo Setup",
|
|
"snippet": "...Complete Kobo e-reader configuration..."
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
### Bad Request (400)
|
|
|
|
Missing or invalid query:
|
|
```json
|
|
{
|
|
"error": "query parameter 'q' is required"
|
|
}
|
|
```
|
|
|
|
Query too short:
|
|
```json
|
|
{
|
|
"error": "query must be at least 2 characters"
|
|
}
|
|
```
|
|
|
|
## Examples
|
|
|
|
```bash
|
|
# Search for Kobo documentation
|
|
curl "https://bookhoard.com/api/docs/search?q=kobo"
|
|
|
|
# Search for API endpoints
|
|
curl "https://bookhoard.com/api/docs/search?q=api%20authentication"
|
|
|
|
# Multi-word search
|
|
curl "https://bookhoard.com/api/docs/search?q=device%20sync"
|
|
```
|
|
|
|
## Notes
|
|
|
|
- Search is case-insensitive
|
|
- Searches across title, section, and content
|
|
- Returns up to all matches (frontend can limit)
|
|
- Snippets include context around matched terms
|
|
- Results are not ranked (order depends on file system traversal)
|
|
```
|
|
|
|
### 2. Update Development Documentation
|
|
|
|
**File**: `docs/contributing/development.md`
|
|
|
|
Add section about the new search endpoint (if there's a section about API development or docs system).
|
|
|
|
### 3. Update README
|
|
|
|
**File**: `README.md`
|
|
|
|
If there's a section about documentation features, mention the new search functionality.
|
|
|
|
---
|
|
|
|
## Implementation Checklist
|
|
|
|
### Phase 1: Backend (1.5 hours)
|
|
- [ ] Create `internal/docs/search.go` with SearchDocuments method
|
|
- [ ] Add search handler to `internal/docs/http_handler.go`
|
|
- [ ] Register search route in `internal/router/docs.go`
|
|
- [ ] Write unit tests for search logic (`internal/docs/search_test.go`)
|
|
- [ ] Run unit tests: `go test ./internal/docs/... -v`
|
|
- [ ] Verify compilation: `go build ./...`
|
|
|
|
### Phase 2: Integration Testing (1 hour)
|
|
- [ ] Create `cmd/server/tests/docs_search_test.go`
|
|
- [ ] Implement table-driven integration tests
|
|
- [ ] Test all three contexts (no auth, user, admin) - though docs search doesn't require auth
|
|
- [ ] Run integration tests: `go test ./cmd/server/tests/... -v`
|
|
- [ ] Verify tests pass
|
|
|
|
### Phase 3: Frontend (1 hour)
|
|
- [ ] Update `web/src/docs.ts` to remove Lunr imports
|
|
- [ ] Replace performDocsSearch with async API call
|
|
- [ ] Test search in browser (manual testing)
|
|
- [ ] Verify no 404 errors for missing JSON files
|
|
|
|
### Phase 4: Cleanup (30 minutes)
|
|
- [ ] Remove Lunr from `package.json`
|
|
- [ ] Run `npm uninstall lunr`
|
|
- [ ] Rebuild frontend: `npm run build:ts`
|
|
- [ ] Verify no TypeScript errors
|
|
- [ ] Remove any references to `/web/src/data/` directory
|
|
|
|
### Phase 5: Documentation & Bruno (30 minutes)
|
|
- [ ] Create `bruno/docs/Search Docs.yml`
|
|
- [ ] Test endpoint with Bruno
|
|
- [ ] Create `docs/developer/api/docs/search.md`
|
|
- [ ] Update `docs/contributing/development.md` if needed
|
|
- [ ] Verify docs render at `/docs` endpoint
|
|
- [ ] Test docs search finds new API documentation
|
|
|
|
### Phase 6: Final Verification (30 minutes)
|
|
- [ ] Run full test suite: `go test ./... -v`
|
|
- [ ] Run verification script: `bash scripts/verify-guidelines.sh`
|
|
- [ ] Build entire project: `go build ./...`
|
|
- [ ] Test search in running application
|
|
- [ ] Verify all three test cases pass:
|
|
- Empty query → 400 error
|
|
- Valid search → results
|
|
- No results → empty array
|
|
- [ ] Check git diff for unintended changes
|
|
- [ ] Review all modified files
|
|
|
|
---
|
|
|
|
## Git Commit Strategy
|
|
|
|
Follow PROJECT_GUIDELINES.md - make multiple logical commits:
|
|
|
|
1. **Backend implementation** (search.go, http_handler.go, router changes)
|
|
2. **Unit tests** (search_test.go)
|
|
3. **Integration tests** (docs_search_test.go)
|
|
4. **Frontend changes** (docs.ts updates, remove Lunr)
|
|
5. **Cleanup** (package.json, remove data directory references)
|
|
6. **Documentation** (API docs, Bruno YAML, contributing guide)
|
|
|
|
---
|
|
|
|
## Success Criteria
|
|
|
|
✅ Backend API `/api/docs/search` returns 200 with results
|
|
✅ Empty/short queries return 400 Bad Request
|
|
✅ Unit tests pass for all search helper functions
|
|
✅ Integration tests pass with table-driven tests
|
|
✅ Bruno YAML validates successfully
|
|
✅ Frontend search works without errors
|
|
✅ No 404s for missing JSON files
|
|
✅ Lunr dependency removed from package.json
|
|
✅ API documentation created and renders
|
|
✅ Full test suite passes
|
|
✅ Verification script passes (0 errors)
|
|
|
|
---
|
|
|
|
## Rollback Plan
|
|
|
|
If issues arise:
|
|
|
|
1. **Backend**: Revert search.go, http_handler.go changes
|
|
2. **Tests**: Remove new test files
|
|
3. **Frontend**: Restore original docs.ts with Lunr code
|
|
4. **Dependencies**: Reinstall Lunr: `npm install lunr@^2.3.9`
|
|
|
|
Use `git diff` to identify changes and `git checkout` to revert specific files if needed.
|