From b3112b1799eb938253d9daaf51b4395b5ed2dcef Mon Sep 17 00:00:00 2001 From: John O'Keefe Date: Sun, 8 Feb 2026 11:05:34 -0500 Subject: [PATCH] docs: add frontend integration guide for tag/contributor normalization Create comprehensive documentation explaining: - Dual-field architecture (display vs search fields) - Normalization rules for tags and contributors - API request/response examples - Frontend implementation guidelines - Search query behavior with examples - Checkbox filter integration - Common mistakes to avoid - Schema reference with indexes - Complete example flows This guide helps frontend developers understand: - How to display normalized tags/contributors - How to implement search functionality - Why there are two sets of fields - Best practices for filter UIs Relates to Tags & Contributors Migration Phase 9 --- docs/FRONTEND_INTEGRATION.md | 191 +++++++++++++++++++++++++++++++++++ 1 file changed, 191 insertions(+) create mode 100644 docs/FRONTEND_INTEGRATION.md diff --git a/docs/FRONTEND_INTEGRATION.md b/docs/FRONTEND_INTEGRATION.md new file mode 100644 index 0000000..ac0e16b --- /dev/null +++ b/docs/FRONTEND_INTEGRATION.md @@ -0,0 +1,191 @@ +# Frontend Integration Notes + +## Tag & Contributor Normalization + +The backend implements dual-field normalization for searchability: + +### Architecture + +| Field Type | Purpose | Behavior | Example | +|-----------|---------|-----------|----------| +| **Display Field** (`tags`, `contributors`) | Show to users | Preserves exact variant, punctuation, proper casing | `"ACME CORP."` | +| **Search Field** (`tags_search`, `contributors_search`) | Search against | Lowercase, no punctuation, deduplicated | `["acme corp"]` | + +### Normalization Rules + +#### Tags +1. Trim whitespace from each tag +2. Titlecase each tag (preserves hyphenation: "non-fiction" → "Non-Fiction") +3. Case-insensitive deduplication +4. Remove punctuation for search field only +5. Store both display and search versions + +#### Contributors +1. Trim whitespace from each contributor +2. Preserve original casing (including CAPSLOCK companies) +3. Preserve original punctuation for display +4. Remove punctuation for search comparison only +5. Case-insensitive deduplication +6. Store both display and search versions + +### API Request/Response + +**Request:** +```json +{ + "tags": ["science-fiction", "ACME CORP.", " O'Reilly Media"], + "contributors": [" Acme Corp ", "acme corp"] +} +``` + +**Response (after normalization):** +```json +{ + "tags": ["Science-Fiction", "O'Reilly Media"], + "tags_search": ["science fiction", "oreilly media"], + "contributors": ["Acme Corp", "acme corp"], + "contributors_search": ["acme corp"] +} +``` + +### Frontend Implementation Guidelines + +#### Display +- Use `tags` and `contributors` fields +- These preserve exact user input (casing, punctuation) +- No transformation needed + +#### Search +- Use search inputs against `tags_search` and `contributors_search` +- Normalize user search input: + - Convert to lowercase + - Remove punctuation (optional but recommended) + - Search using `= ANY()` operator + +#### User Typing "Science-Fiction" +```typescript +// User types exact value +const searchValue = "Science-Fiction"; + +// Backend normalizes to display and search versions +// Display: "Science-Fiction" +// Search: "science fiction" +``` + +#### Search Query Behavior +```typescript +// User searches: "ACME CORP." +// Backend normalizes search to: "acme corp" +// This matches contributors_search = ["acme corp"] +// Which finds display contributors = ["ACME CORP.", "Acme Corp", "acme corp"] +``` + +### Checkbox Filter Integration + +When building frontend checkbox filters for contributors/tags: + +#### Get Unique Values for Dropdown +```typescript +// Fetch distinct normalized values for filters +GET /api/contributors?distinct=true +Response: ["acme corp", "oreilly media", "penguin"] + +// Render as checkboxes (using display names from another endpoint or mapping) +``` + +#### Filter Query +```typescript +// User selects checkbox +const filterValue = "acme corp"; + +// API request (filter by search field) +{ + "contributors_search": ["acme corp"] +} + +// Backend matches contributors_search array using = ANY() +``` + +### Important Notes + +1. **Display ≠ Search**: Always send search queries to search fields, not display fields +2. **Backend Normalization**: Backend normalizes input on CREATE/UPDATE, so always use search fields for filtering +3. **Case Sensitivity**: Search is case-insensitive, display is case-preserved +4. **Punctuation**: Display preserves it, search ignores it +5. **Deduplication**: Search fields are deduplicated, display fields are not + +### Common Mistakes to Avoid + +❌ **Searching display field directly** +```typescript +// WRONG - Will miss different casing/punctuation +WHERE 'ACME CORP.' = ANY(contributors) +``` + +✅ **Search search field** +```typescript +// CORRECT - Case-insensitive, punctuation-free +WHERE 'acme corp' = ANY(contributors_search) +``` + +❌ **Don't normalize user search input** +```typescript +// WRONG - If user types "ACME CORP" explicitly to find exact match +const search = "acme corp"; // Changes user's intent +``` + +✅ **Use exact user input for search** +```typescript +// CORRECT - Backend handles normalization +const search = "ACME CORP"; // Backend will match "acme corp" in search field +``` + +### Schema Reference + +**Display Fields:** +- `tags TEXT[]` - Titlecase, original punctuation +- `contributors TEXT[]` - Original casing, original punctuation + +**Search Fields:** +- `tags_search TEXT[]` - Lowercase, no punctuation, deduplicated +- `contributors_search TEXT[]` - Lowercase, no punctuation, deduplicated + +**GIN Indexes:** +- `idx_media_items_tags_search` - Fast search on tags_search +- `idx_media_items_contributors_search` - Fast search on contributors_search +- `idx_media_items_tags_gin` - Display field (if needed) +- `idx_media_items_contributors_gin` - Display field (if needed) + +### Example Flow + +1. **User creates media item:** + - Input: `tags: ["science-fiction", "ACME CORP."]` + - Backend stores: + - `tags`: `["Science-Fiction"]` (titlecased) + - `tags_search`: `["science fiction"]` (lowercase, no punctuation) + - `contributors`: `["ACME CORP."]` (preserved) + - `contributors_search`: `["acme corp"]` (normalized) + +2. **User searches "ACME CORP":** + - Frontend sends: `q: "ACME CORP"` + - Backend searches `tags_search` and `contributors_search` + - Finds: `contributors_search = ["acme corp"]` → MATCH ✅ + - Returns: Media item with `contributors = ["ACME CORP."]` + +3. **User searches "acme corp":** + - Frontend sends: `q: "acme corp"` + - Backend searches `tags_search` and `contributors_search` + - Finds: `contributors_search = ["acme corp"]` → MATCH ✅ + - Returns: Media item with `contributors = ["ACME CORP."]` + +4. **User searches "science-fiction":** + - Frontend sends: `q: "science-fiction"` + - Backend searches `tags_search` + - Finds: `tags_search = ["science fiction"]` → NO MATCH (hyphen vs space) + - Does NOT return (but fuzzy search might catch it) + +5. **User searches "science fiction":** + - Frontend sends: `q: "science fiction"` + - Backend searches `tags_search` + - Finds: `tags_search = ["science fiction"]` → MATCH ✅ + - Returns: Media item with `tags = ["Science-Fiction"]`