docs: add frontend integration guide for tag/contributor normalization
Create comprehensive documentation explaining: - Dual-field architecture (display vs search fields) - Normalization rules for tags and contributors - API request/response examples - Frontend implementation guidelines - Search query behavior with examples - Checkbox filter integration - Common mistakes to avoid - Schema reference with indexes - Complete example flows This guide helps frontend developers understand: - How to display normalized tags/contributors - How to implement search functionality - Why there are two sets of fields - Best practices for filter UIs Relates to Tags & Contributors Migration Phase 9
This commit is contained in:
@@ -0,0 +1,191 @@
|
||||
# Frontend Integration Notes
|
||||
|
||||
## Tag & Contributor Normalization
|
||||
|
||||
The backend implements dual-field normalization for searchability:
|
||||
|
||||
### Architecture
|
||||
|
||||
| Field Type | Purpose | Behavior | Example |
|
||||
|-----------|---------|-----------|----------|
|
||||
| **Display Field** (`tags`, `contributors`) | Show to users | Preserves exact variant, punctuation, proper casing | `"ACME CORP."` |
|
||||
| **Search Field** (`tags_search`, `contributors_search`) | Search against | Lowercase, no punctuation, deduplicated | `["acme corp"]` |
|
||||
|
||||
### Normalization Rules
|
||||
|
||||
#### Tags
|
||||
1. Trim whitespace from each tag
|
||||
2. Titlecase each tag (preserves hyphenation: "non-fiction" → "Non-Fiction")
|
||||
3. Case-insensitive deduplication
|
||||
4. Remove punctuation for search field only
|
||||
5. Store both display and search versions
|
||||
|
||||
#### Contributors
|
||||
1. Trim whitespace from each contributor
|
||||
2. Preserve original casing (including CAPSLOCK companies)
|
||||
3. Preserve original punctuation for display
|
||||
4. Remove punctuation for search comparison only
|
||||
5. Case-insensitive deduplication
|
||||
6. Store both display and search versions
|
||||
|
||||
### API Request/Response
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"tags": ["science-fiction", "ACME CORP.", " O'Reilly Media"],
|
||||
"contributors": [" Acme Corp ", "acme corp"]
|
||||
}
|
||||
```
|
||||
|
||||
**Response (after normalization):**
|
||||
```json
|
||||
{
|
||||
"tags": ["Science-Fiction", "O'Reilly Media"],
|
||||
"tags_search": ["science fiction", "oreilly media"],
|
||||
"contributors": ["Acme Corp", "acme corp"],
|
||||
"contributors_search": ["acme corp"]
|
||||
}
|
||||
```
|
||||
|
||||
### Frontend Implementation Guidelines
|
||||
|
||||
#### Display
|
||||
- Use `tags` and `contributors` fields
|
||||
- These preserve exact user input (casing, punctuation)
|
||||
- No transformation needed
|
||||
|
||||
#### Search
|
||||
- Use search inputs against `tags_search` and `contributors_search`
|
||||
- Normalize user search input:
|
||||
- Convert to lowercase
|
||||
- Remove punctuation (optional but recommended)
|
||||
- Search using `= ANY()` operator
|
||||
|
||||
#### User Typing "Science-Fiction"
|
||||
```typescript
|
||||
// User types exact value
|
||||
const searchValue = "Science-Fiction";
|
||||
|
||||
// Backend normalizes to display and search versions
|
||||
// Display: "Science-Fiction"
|
||||
// Search: "science fiction"
|
||||
```
|
||||
|
||||
#### Search Query Behavior
|
||||
```typescript
|
||||
// User searches: "ACME CORP."
|
||||
// Backend normalizes search to: "acme corp"
|
||||
// This matches contributors_search = ["acme corp"]
|
||||
// Which finds display contributors = ["ACME CORP.", "Acme Corp", "acme corp"]
|
||||
```
|
||||
|
||||
### Checkbox Filter Integration
|
||||
|
||||
When building frontend checkbox filters for contributors/tags:
|
||||
|
||||
#### Get Unique Values for Dropdown
|
||||
```typescript
|
||||
// Fetch distinct normalized values for filters
|
||||
GET /api/contributors?distinct=true
|
||||
Response: ["acme corp", "oreilly media", "penguin"]
|
||||
|
||||
// Render as checkboxes (using display names from another endpoint or mapping)
|
||||
```
|
||||
|
||||
#### Filter Query
|
||||
```typescript
|
||||
// User selects checkbox
|
||||
const filterValue = "acme corp";
|
||||
|
||||
// API request (filter by search field)
|
||||
{
|
||||
"contributors_search": ["acme corp"]
|
||||
}
|
||||
|
||||
// Backend matches contributors_search array using = ANY()
|
||||
```
|
||||
|
||||
### Important Notes
|
||||
|
||||
1. **Display ≠ Search**: Always send search queries to search fields, not display fields
|
||||
2. **Backend Normalization**: Backend normalizes input on CREATE/UPDATE, so always use search fields for filtering
|
||||
3. **Case Sensitivity**: Search is case-insensitive, display is case-preserved
|
||||
4. **Punctuation**: Display preserves it, search ignores it
|
||||
5. **Deduplication**: Search fields are deduplicated, display fields are not
|
||||
|
||||
### Common Mistakes to Avoid
|
||||
|
||||
❌ **Searching display field directly**
|
||||
```typescript
|
||||
// WRONG - Will miss different casing/punctuation
|
||||
WHERE 'ACME CORP.' = ANY(contributors)
|
||||
```
|
||||
|
||||
✅ **Search search field**
|
||||
```typescript
|
||||
// CORRECT - Case-insensitive, punctuation-free
|
||||
WHERE 'acme corp' = ANY(contributors_search)
|
||||
```
|
||||
|
||||
❌ **Don't normalize user search input**
|
||||
```typescript
|
||||
// WRONG - If user types "ACME CORP" explicitly to find exact match
|
||||
const search = "acme corp"; // Changes user's intent
|
||||
```
|
||||
|
||||
✅ **Use exact user input for search**
|
||||
```typescript
|
||||
// CORRECT - Backend handles normalization
|
||||
const search = "ACME CORP"; // Backend will match "acme corp" in search field
|
||||
```
|
||||
|
||||
### Schema Reference
|
||||
|
||||
**Display Fields:**
|
||||
- `tags TEXT[]` - Titlecase, original punctuation
|
||||
- `contributors TEXT[]` - Original casing, original punctuation
|
||||
|
||||
**Search Fields:**
|
||||
- `tags_search TEXT[]` - Lowercase, no punctuation, deduplicated
|
||||
- `contributors_search TEXT[]` - Lowercase, no punctuation, deduplicated
|
||||
|
||||
**GIN Indexes:**
|
||||
- `idx_media_items_tags_search` - Fast search on tags_search
|
||||
- `idx_media_items_contributors_search` - Fast search on contributors_search
|
||||
- `idx_media_items_tags_gin` - Display field (if needed)
|
||||
- `idx_media_items_contributors_gin` - Display field (if needed)
|
||||
|
||||
### Example Flow
|
||||
|
||||
1. **User creates media item:**
|
||||
- Input: `tags: ["science-fiction", "ACME CORP."]`
|
||||
- Backend stores:
|
||||
- `tags`: `["Science-Fiction"]` (titlecased)
|
||||
- `tags_search`: `["science fiction"]` (lowercase, no punctuation)
|
||||
- `contributors`: `["ACME CORP."]` (preserved)
|
||||
- `contributors_search`: `["acme corp"]` (normalized)
|
||||
|
||||
2. **User searches "ACME CORP":**
|
||||
- Frontend sends: `q: "ACME CORP"`
|
||||
- Backend searches `tags_search` and `contributors_search`
|
||||
- Finds: `contributors_search = ["acme corp"]` → MATCH ✅
|
||||
- Returns: Media item with `contributors = ["ACME CORP."]`
|
||||
|
||||
3. **User searches "acme corp":**
|
||||
- Frontend sends: `q: "acme corp"`
|
||||
- Backend searches `tags_search` and `contributors_search`
|
||||
- Finds: `contributors_search = ["acme corp"]` → MATCH ✅
|
||||
- Returns: Media item with `contributors = ["ACME CORP."]`
|
||||
|
||||
4. **User searches "science-fiction":**
|
||||
- Frontend sends: `q: "science-fiction"`
|
||||
- Backend searches `tags_search`
|
||||
- Finds: `tags_search = ["science fiction"]` → NO MATCH (hyphen vs space)
|
||||
- Does NOT return (but fuzzy search might catch it)
|
||||
|
||||
5. **User searches "science fiction":**
|
||||
- Frontend sends: `q: "science fiction"`
|
||||
- Backend searches `tags_search`
|
||||
- Finds: `tags_search = ["science fiction"]` → MATCH ✅
|
||||
- Returns: Media item with `tags = ["Science-Fiction"]`
|
||||
Reference in New Issue
Block a user