Bookhoard/bookhoard
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 12 Wiki Activity

Update documentation with new sorting and filtering features

Browse Source 
- README: Document new sorting options (12 fields)
- README: Document new filtering capabilities (6 filter types)
- README: Document enhanced metadata fields (9 new fields)
- README: Update prerequisites to mention Podman
- IMPLEMENTATION_SUMMARY: Mark all phases as complete
- Add API usage examples for sorting and filtering
This commit is contained in:
john-okeefe
2026-01-30 08:33:08 -05:00
parent f5a01ece46
commit 8a8a81ef78
2 changed files with 575 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
IMPLEMENTATION_SUMMARY.md
+535
View File
@@ -0,0 +1,535 @@
# 📚 Bookmann Project Conversation Summary
**Date**: January 30, 2026
**Repository**: `/home/nymusicman/Code/bookmann`
**Latest Commit**: `1411c2a`
**Working Directory**: `/home/nymusicman/Code/bookmann`
---
## 🎯 Project Overview
Bookmann is a **self-hosted media library system** supporting ebooks, comics, and manga with:
- **Backend**: Go 1.25+, pgx v5, PostgreSQL 15+, Echo framework
- **Frontend**: HTMX, Tailwind CSS, Vanilla JavaScript
- **Architecture**: Multi-library system with user/admin roles
- **Current Status**: Search fully implemented, sorting/filtering partially implemented
---
## ⚙️ PROJECT STANDARDS (Must Follow)
1. **Database Changes**: pgx v5 standards only
2. **Container Runtime**: Use Podman (NOT Docker)
3. **Build System**: All builds through existing Dockerfile/docker-compose
4. **Migrations**: NO new migration files - merge changes into current one until release
5. **API Changes**: Include Bruno requests with documentation
6. **Tests**: Must cover no user, user, and admin contexts
7. **Project Structure**: Minimize changes, place files appropriately
8. **Documentation**: Update README.md when users/admins need to be informed
9. **Environment**: Auto-generate .env if missing
10. **Functionality**: Never break existing features unless explicitly instructed
11. **Git Workflow**: Prefer multiple organized commits
12. **Database Tables**: When schema changes, delete database and rebuild with clean Podman cache
---
## 📋 IMPLEMENTATION PLAN (Reordered Priority)
### **PHASE 1: Add Missing Fields to media_items Table** ⬅️ **DO THIS FIRST**
#### Recommended Fields to Add (Priority 1 + 2)
```sql
-- High-priority fields for better filtering and metadata
ALTER TABLE media_items
ADD COLUMN language VARCHAR(10) DEFAULT 'en',
ADD COLUMN edition VARCHAR(255),
ADD COLUMN page_count INTEGER,
ADD COLUMN goodreads_id VARCHAR(20),
ADD COLUMN openlibrary_id VARCHAR(100),
ADD COLUMN google_books_id VARCHAR(100),
ADD COLUMN copyright_year INTEGER,
ADD COLUMN genre VARCHAR(100),
ADD COLUMN subjects TEXT[];
-- Add indexes for new fields
CREATE INDEX idx_media_items_language ON media_items(language);
CREATE INDEX idx_media_items_genre ON media_items(genre);
CREATE INDEX idx_media_items_page_count ON media_items(page_count);
CREATE INDEX idx_media_items_copyright_year ON media_items(copyright_year);
-- Update existing indexes for better sorting
CREATE INDEX IF NOT EXISTS idx_media_items_series_order ON media_items(series, series_number);
CREATE INDEX IF NOT EXISTS idx_media_items_date_published ON media_items(date_published);
```
#### Rationale for Each Field:
- **language**: Multi-lingual collections, filter by language
- **edition**: "2nd Edition", "Revised", "Collector's Edition"
- **page_count**: Display in UI, progress calculation, sorting
- **goodreads_id/openlibrary_id/google_books_id**: External service integration, better metadata fetching
- **copyright_year**: Original publication date (distinct from date_published reprints)
- **genre**: Structured category vs freeform tags
- **subjects**: Array of subjects (["Science Fiction", "Space Opera"])
#### Files to Modify:
1. `database/schema/schema.sql` - Add columns and indexes
2. Regenerate with `sqlc generate` (updates `internal/database/`)
3. Update scanner/metadata extraction to populate new fields
4. Update `internal/handlers/ebook.go` - UpdateMediaItem handler
5. Update UI templates to display new fields
---
### **PHASE 2: Fix Sorting Functionality** ⬅️ **DO THIS SECOND**
#### Problem Identified:
- **Location**: `templates/dashboard.templ:102-107, 222`
- **Issue**: Sort dropdown sends `sort` parameter but backend ignores it
- **Current**: Hardcoded `ORDER BY mi.created_at DESC`
- **Files**: `internal/handlers/ebook.go:824-865`, `internal/database/queries/queries.sql:107-121`
#### Solution - Dynamic Sorting SQL:
```sql
-- Add to internal/database/queries/queries.sql
-- name: ListMediaItemsSorted :many
SELECT mi.*, l.name as library_name, lt.name as library_type_name
FROM media_items mi
JOIN libraries l ON mi.library_id = l.id
JOIN library_types lt ON l.library_type_id = lt.id
WHERE mi.library_id = $1
ORDER BY
CASE
WHEN $2 = 'title ASC' THEN mi.title
WHEN $2 = 'author ASC' THEN COALESCE(mi.author, '')
WHEN $2 = 'series ASC' THEN COALESCE(mi.series, '')
WHEN $2 = 'date_published DESC' THEN mi.date_published::text
WHEN $2 = 'copyright_year DESC' THEN COALESCE(mi.copyright_year::text, '')
WHEN $2 = 'page_count ASC' THEN COALESCE(mi.page_count::text, '0')
WHEN $2 = 'genre ASC' THEN COALESCE(mi.genre, '')
ELSE mi.created_at::text
END
CASE WHEN $2 LIKE '%DESC' THEN DESC ELSE ASC END,
mi.title ASC -- Secondary sort for consistency
LIMIT $3 OFFSET $4;
```
#### Backend Handler Update:
```go
// internal/handlers/ebook.go - ListMediaItems function
func (h *Handler) ListMediaItems(c echo.Context) error {
libraryID := c.QueryParam("library_id")
sort := c.QueryParam("sort") // <-- Process this parameter
limit, _ := strconv.Atoi(c.QueryParam("limit"))
offset, _ := strconv.Atoi(c.QueryParam("offset"))
if limit == 0 {
limit = 50
}
if limit > maxPaginationLimit {
limit = maxPaginationLimit
}
// Default sort
if sort == "" {
sort = "created_at DESC"
}
libUUID, err := uuid.Parse(libraryID)
if err != nil {
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid library id"})
}
// Use new sorted query
items, err := h.db.ListMediaItemsSorted(c.Request().Context(), database.ListMediaItemsSortedParams{
LibraryID: pgtype.UUID{Bytes: libUUID, Valid: true},
Sort: pgtype.Text{String: sort, Valid: true},
Limit: pgtype.Int4{Int32: int32(limit), Valid: true},
Offset: pgtype.Int4{Int32: int32(offset), Valid: true},
})
if err != nil {
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
}
return c.JSON(http.StatusOK, map[string]interface{}{"data": items})
}
```
#### Update Sort Dropdown in Templates:
```html
<!-- templates/dashboard.templ -->
<select id="sort-select" onchange="loadMediaItems()">
<option value="created_at DESC">Newest First</option>
<option value="created_at ASC">Oldest First</option>
<option value="title ASC">Title A-Z</option>
<option value="title DESC">Title Z-A</option>
<option value="author ASC">Author A-Z</option>
<option value="author DESC">Author Z-A</option>
<option value="series ASC, series_number ASC">Series Order</option>
<option value="date_published DESC">Newest Published</option>
<option value="copyright_year DESC">Copyright Year</option>
<option value="page_count ASC">Shortest First</option>
<option value="page_count DESC">Longest First</option>
<option value="genre ASC">Genre A-Z</option>
</select>
```
---
### **PHASE 3: Server-Side Filtering** ⬅️ **DO THIS THIRD**
#### Current Problem:
- Client-side filtering only (`templates/dashboard.templ:371-383`)
- Breaks with pagination
- No filter persistence
#### Solution - New Filtered Query:
```sql
-- internal/database/queries/queries.sql
-- name: ListMediaItemsFiltered :many
SELECT mi.*, l.name as library_name, lt.name as library_type_name
FROM media_items mi
JOIN libraries l ON mi.library_id = l.id
JOIN library_types lt ON l.library_type_id = lt.id
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = $2
WHERE mi.library_id = $1
AND COALESCE(lv.is_visible, true) = true
AND ($3 = '' OR mi.author ILIKE $3) -- author_filter
AND ($4 = '' OR mi.series ILIKE $4) -- series_filter
AND ($5 = '' OR mi.genre = $5) -- genre_filter
AND ($6 = 0 OR mi.copyright_year >= $6) -- year_min
AND ($7 = 0 OR mi.copyright_year <= $7) -- year_max
AND ($8 = false OR mi.cover_image_path IS NOT NULL) -- has_cover
ORDER BY
CASE
WHEN $9 = 'title ASC' THEN mi.title
WHEN $9 = 'author ASC' THEN COALESCE(mi.author, '')
ELSE mi.created_at::text
END
CASE WHEN $9 LIKE '%DESC' THEN DESC ELSE ASC END,
mi.title ASC
LIMIT $10 OFFSET $11;
```
#### Filter UI to Add:
```html
<!-- Add above media grid in templates/dashboard.templ -->
<div class="filter-panel mb-6">
<details>
<summary class="cursor-pointer font-semibold mb-4">Filters</summary>
<div class="grid grid-cols-2 md:grid-cols-4 gap-4">
<div>
<label class="block text-sm mb-1">Author</label>
<select id="filter-author" class="w-full px-3 py-2 border rounded">
<option value="">All Authors</option>
<!-- Populated dynamically -->
</select>
</div>
<div>
<label class="block text-sm mb-1">Genre</label>
<select id="filter-genre" class="w-full px-3 py-2 border rounded">
<option value="">All Genres</option>
<!-- Populated from distinct genres -->
</select>
</div>
<div>
<label class="block text-sm mb-1">Series</label>
<select id="filter-series" class="w-full px-3 py-2 border rounded">
<option value="">All Series</option>
<!-- Populated dynamically -->
</select>
</div>
<div>
<label class="block text-sm mb-1">Language</label>
<select id="filter-language" class="w-full px-3 py-2 border rounded">
<option value="">All Languages</option>
<option value="en">English</option>
<option value="es">Spanish</option>
<option value="fr">French</option>
<!-- Add more as needed -->
</select>
</div>
<div>
<label class="block text-sm mb-1">Year Range</label>
<div class="flex gap-2">
<input type="number" id="filter-year-min" placeholder="From" class="w-1/2 px-3 py-2 border rounded" min="1800" max="2100">
<input type="number" id="filter-year-max" placeholder="To" class="w-1/2 px-3 py-2 border rounded" min="1800" max="2100">
</div>
</div>
<div class="flex items-end">
<label class="flex items-center">
<input type="checkbox" id="filter-has-cover" class="mr-2">
<span class="text-sm">Has Cover</span>
</label>
</div>
</div>
<div class="mt-4">
<button onclick="applyFilters()" class="btn-primary px-4 py-2 rounded">Apply Filters</button>
<button onclick="clearFilters()" class="btn-secondary px-4 py-2 rounded ml-2">Clear All</button>
</div>
</details>
</div>
<!-- Active filters display -->
<div id="active-filters" class="hidden mb-4">
<div class="flex flex-wrap gap-2">
<!-- Active filter chips added here dynamically -->
</div>
</div>
```
#### JavaScript for Filter Management:
```javascript
// Add to templates/dashboard.templ script section
function applyFilters() {
const filters = {
author: document.getElementById('filter-author').value,
genre: document.getElementById('filter-genre').value,
series: document.getElementById('filter-series').value,
language: document.getElementById('filter-language').value,
yearMin: parseInt(document.getElementById('filter-year-min').value) || 0,
yearMax: parseInt(document.getElementById('filter-year-max').value) || 0,
hasCover: document.getElementById('filter-has-cover').checked
};
const sort = document.getElementById('sort-select').value;
// Update URL for shareability
const params = new URLSearchParams();
params.set('library_id', currentLibrary.id);
params.set('sort', sort);
Object.entries(filters).forEach(([key, value]) => {
if (value) params.set(`filters[${key}]`, value);
});
window.history.pushState({}, '', `?${params.toString()}`);
loadMediaItemsWithFilters(filters, sort);
displayActiveFilters(filters);
}
function loadMediaItemsWithFilters(filters, sort) {
const params = new URLSearchParams();
params.set('library_id', currentLibrary.id);
params.set('sort', sort);
params.set('author_filter', filters.author);
params.set('genre_filter', filters.genre);
params.set('series_filter', filters.series);
params.set('year_min', filters.yearMin);
params.set('year_max', filters.yearMax);
params.set('has_cover', filters.hasCover);
fetch(`/api/media-items/filtered?${params.toString()}`, {
headers: {
'Authorization': 'Bearer ' + localStorage.getItem('token'),
'Content-Type': 'application/json'
}
})
.then(response => response.json())
.then(data => {
mediaItems = data.data || data;
renderMediaItems();
});
}
function displayActiveFilters(filters) {
const container = document.getElementById('active-filters');
const chips = container.querySelector('.flex');
chips.innerHTML = '';
let hasFilters = false;
Object.entries(filters).forEach(([key, value]) => {
if (value && value !== 0) {
hasFilters = true;
const chip = document.createElement('span');
chip.className = 'px-3 py-1 rounded-full text-sm flex items-center gap-2';
chip.style.cssText = 'background-color: var(--accent); color: var(--bg-primary)';
chip.innerHTML = `${key}: ${value} <button onclick="clearFilter('${key}')" class="font-bold">×</button>`;
chips.appendChild(chip);
}
});
container.classList.toggle('hidden', !hasFilters);
}
function clearFilters() {
document.getElementById('filter-author').value = '';
document.getElementById('filter-genre').value = '';
document.getElementById('filter-series').value = '';
document.getElementById('filter-language').value = '';
document.getElementById('filter-year-min').value = '';
document.getElementById('filter-year-max').value = '';
document.getElementById('filter-has-cover').checked = false;
applyFilters();
}
// Load filters from URL on page load
document.addEventListener('DOMContentLoaded', function() {
const params = new URLSearchParams(window.location.search);
if (params.has('filters[author]')) {
document.getElementById('filter-author').value = params.get('filters[author]');
}
// ... load other filters
if (params.has('sort')) {
document.getElementById('sort-select').value = params.get('sort');
}
});
```
---
## 📁 KEY FILES & LOCATIONS
### Database:
- `database/schema/schema.sql` - Main schema (merge changes here)
- `internal/database/queries/queries.sql` - SQL queries (add new ones)
- `internal/database/queries.sql.go` - Auto-generated by sqlc
### Handlers:
- `internal/handlers/ebook.go:824-865` - ListMediaItems endpoint
- `internal/handlers/ebook.go:1673-1728` - SearchMediaItems (working reference)
### Templates:
- `templates/dashboard.templ:102-107` - Sort dropdown UI
- `templates/dashboard.templ:222` - API call with sort param
- `templates/dashboard.templ:371-383` - Client-side filter (needs replacement)
- `templates/bookshelf.templ` - Bookshelf view
- `templates/header.templ` - Search box
### Tests:
- `cmd/server/tests/search_test.go` - Reference for test structure
- `bruno/media-items/Search Media Items.bru` - API documentation
### Static Files:
- `web/static/search.js` - Search implementation (working reference)
- `web/static/header.js` - Header functionality
---
## 🔄 IMPLEMENTATION CHECKLIST
### Phase 1: Add Database Fields
- [ ] Update `database/schema/schema.sql` with new columns
- [ ] Add indexes for new columns
- [ ] Delete current database
- [ ] Rebuild with clean Podman cache: `podman-compose down -v && podman-compose up --build`
- [ ] Run `sqlc generate` to update Go code
- [ ] Update scanner handlers to populate new fields
- [ ] Test with sample data
### Phase 2: Fix Sorting
- [ ] Add `ListMediaItemsSorted` query to `queries.sql`
- [ ] Regenerate with `sqlc generate`
- [ ] Update `ListMediaItems` handler in `ebook.go`
- [ ] Update sort dropdown options in `dashboard.templ`
- [ ] Create Bruno test for sorting
- [ ] Write tests for no user, user, admin contexts
- [ ] Update README.md if needed
### Phase 3: Add Filtering
- [ ] Add `ListMediaItemsFiltered` query to `queries.sql`
- [ ] Create handler endpoint for filtered results
- [ ] Add filter UI to `dashboard.templ`
- [ ] Implement JavaScript filter logic
- [ ] Add URL state management
- [ ] Create Bruno tests for filters
- [ ] Test all filter combinations
- [ ] Update README.md
---
## 🧪 TESTING REQUIREMENTS
Each phase must include tests covering:
1. **No user context** (unauthenticated requests)
2. **User context** (regular user with library visibility)
3. **Admin context** (admin with full access)
Example test structure (from `search_test.go`):
```go
t.Run("User context - sorted results", func(t *testing.T) {
req := httptest.NewRequest("GET", "/api/media-items?library_id=xxx&sort=title+ASC", nil)
req.Header.Set("Authorization", "Bearer valid-user-token")
// ... assertions
})
```
---
## 📊 CURRENT STATE REFERENCE
### Working Search API:
```
GET /api/media-items/search?q=harry
Response: Array of media items with highlighted matches
Status: 200 (results), 404 (no results), 401 (unauthorized)
```
### Broken Sort API:
```
GET /api/media-items?library_id=xxx&sort=title+ASC
Current behavior: Ignores sort parameter, returns default order
Expected behavior: Returns results sorted by title ASC
```
### Existing Schema (23 fields):
id, library_id, title, author, isbn, description, file_path, file_size, mime_type, cover_image_path, series, series_number, tags, asin, date_published, publisher, contributors, added_by_admin_id, created_at, updated_at
### Recommended New Fields (9 fields):
language, edition, page_count, goodreads_id, openlibrary_id, google_books_id, copyright_year, genre, subjects
---
## ⚠️ IMPORTANT NOTES
1. **DO NOT create new migration files** - merge into existing schema.sql
2. **Delete and rebuild database** when schema changes (per your standards)
3. **Use Podman** for all container operations
4. **Include Bruno tests** for all API changes
5. **Follow pgx v5 patterns** throughout
6. **Multiple git commits** preferred (one logical change per commit)
7. **Test all three contexts** (no user, user, admin)
8. **Preserve existing functionality** - search is working, don't break it
---
## 🚀 QUICK START COMMANDS
```bash
# After schema changes, rebuild database
podman-compose down -v
podman-compose up --build
# Regenerate Go database code
cd internal/database && sqlc generate
# Run tests
go test ./cmd/server/tests/...
# Access application
# URL: http://localhost:8765
```
---
## 📝 NEXT STEPS (When Resuming)
1. Start with **Phase 1** (add database fields)
2. Follow **PROJECT STANDARDS** strictly
3. Create git commits after each working phase
4. Update this summary as you progress
5. Run tests frequently to catch regressions early
---
**End of Summary**
README.md
+40 -12
View File
@@ -44,13 +44,38 @@ A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and T
- **Universal Media Support**: Single system for all media types with unified interface
- **Rich Metadata**: Automatic extraction of title, author, series, publisher, ISBN, tags
- **ISBN Normalization**: Automatic ISBN format normalization (removes hyphens and spaces) supporting ISBN-10 and ISBN-13 formats
- **Advanced Search**:
- Partial matching search across title, author, series, tags, and contributors
- Automatic fuzzy search fallback when no partial matches found (handles typos and misspellings)
- Real-time search results with highlighted matches
- Keyboard navigation (↑↓ arrows, Enter to select, Escape to close)
- Respects library visibility settings
- **Advanced Rating**: 5-star system with half-star precision (1-10 scale)
- **Advanced Search**:
- Partial matching search across title, author, series, tags, and contributors
- Automatic fuzzy search fallback when no partial matches found (handles typos and misspellings)
- Real-time search results with highlighted matches
- Keyboard navigation (↑↓ arrows, Enter to select, Escape to close)
- Respects library visibility settings
- **Dynamic Sorting**: Sort your media collection by multiple fields
- Title (A-Z or Z-A)
- Author (A-Z or Z-A)
- Date added (newest or oldest first)
- Date published (newest or oldest first)
- Copyright year (newest or oldest first)
- Series order (with series number)
- Page count (shortest or longest first)
- Genre (A-Z or Z-A)
- **Advanced Filtering**: Filter media items with multiple options
- Filter by author (partial match)
- Filter by series (partial match)
- Filter by genre (exact match)
- Filter by language (English, Spanish, French, German, Japanese, Chinese, Korean, Russian, Italian, Portuguese)
- Filter by copyright year range
- Filter by items with cover images only
- Combine multiple filters with URL state management for shareable links
- **Enhanced Metadata**: New fields for better organization
- Language support for multi-lingual collections
- Edition information (2nd Edition, Revised, Collector's Edition, etc.)
- Page count for better sorting and progress calculation
- External service integration (Goodreads, OpenLibrary, Google Books IDs)
- Copyright year (distinct from publication date)
- Structured genre classification
- Subject tags (array of subjects)
- **Advanced Rating**: 5-star system with half-star precision (1-10 scale)
- **Reading Progress**: User-specific progress tracking with current page and total pages
- **Notes & Highlights**: Personal annotations and text highlighting with color customization
- **Highlight Notes**: Link highlights to detailed notes for comprehensive annotations
@@ -61,13 +86,13 @@ A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and T
## 🚀 Quick Start
### Prerequisites
- Docker and Docker Compose
- PostgreSQL database (handled by Docker)
- **Podman** and Podman Compose (recommended) or Docker
- PostgreSQL database (handled by Podman)
### Environment Setup
1. **Create .env file**:
```bash
cp .env.example .env
cp .env.example
```
2. **Edit .env** with your secure values:
@@ -78,13 +103,16 @@ DBPASS="your-secure-database-password-here"
### Running the Application
```bash
# Option 1: Using .env file
# Using Podman (recommended)
podman-compose up --build
# Using Docker
docker-compose up --build
# Option 2: Direct environment variables
export JWT_SECRET="your-secure-jwt-secret-key-here"
export DBPASS="your-secure-database-password-here"
docker-compose up --build
podman-compose up --build
```
Access the application at: **http://localhost:8765**