diff --git a/IMPLEMENTATION_SUMMARY.md b/IMPLEMENTATION_SUMMARY.md new file mode 100644 index 0000000..abd6488 --- /dev/null +++ b/IMPLEMENTATION_SUMMARY.md @@ -0,0 +1,535 @@ +# ๐Ÿ“š Bookmann Project Conversation Summary + +**Date**: January 30, 2026 +**Repository**: `/home/nymusicman/Code/bookmann` +**Latest Commit**: `1411c2a` +**Working Directory**: `/home/nymusicman/Code/bookmann` + +--- + +## ๐ŸŽฏ Project Overview + +Bookmann is a **self-hosted media library system** supporting ebooks, comics, and manga with: +- **Backend**: Go 1.25+, pgx v5, PostgreSQL 15+, Echo framework +- **Frontend**: HTMX, Tailwind CSS, Vanilla JavaScript +- **Architecture**: Multi-library system with user/admin roles +- **Current Status**: Search fully implemented, sorting/filtering partially implemented + +--- + +## โš™๏ธ PROJECT STANDARDS (Must Follow) + +1. **Database Changes**: pgx v5 standards only +2. **Container Runtime**: Use Podman (NOT Docker) +3. **Build System**: All builds through existing Dockerfile/docker-compose +4. **Migrations**: NO new migration files - merge changes into current one until release +5. **API Changes**: Include Bruno requests with documentation +6. **Tests**: Must cover no user, user, and admin contexts +7. **Project Structure**: Minimize changes, place files appropriately +8. **Documentation**: Update README.md when users/admins need to be informed +9. **Environment**: Auto-generate .env if missing +10. **Functionality**: Never break existing features unless explicitly instructed +11. **Git Workflow**: Prefer multiple organized commits +12. **Database Tables**: When schema changes, delete database and rebuild with clean Podman cache + +--- + +## ๐Ÿ“‹ IMPLEMENTATION PLAN (Reordered Priority) + +### **PHASE 1: Add Missing Fields to media_items Table** โฌ…๏ธ **DO THIS FIRST** + +#### Recommended Fields to Add (Priority 1 + 2) + +```sql +-- High-priority fields for better filtering and metadata +ALTER TABLE media_items + ADD COLUMN language VARCHAR(10) DEFAULT 'en', + ADD COLUMN edition VARCHAR(255), + ADD COLUMN page_count INTEGER, + ADD COLUMN goodreads_id VARCHAR(20), + ADD COLUMN openlibrary_id VARCHAR(100), + ADD COLUMN google_books_id VARCHAR(100), + ADD COLUMN copyright_year INTEGER, + ADD COLUMN genre VARCHAR(100), + ADD COLUMN subjects TEXT[]; + +-- Add indexes for new fields +CREATE INDEX idx_media_items_language ON media_items(language); +CREATE INDEX idx_media_items_genre ON media_items(genre); +CREATE INDEX idx_media_items_page_count ON media_items(page_count); +CREATE INDEX idx_media_items_copyright_year ON media_items(copyright_year); + +-- Update existing indexes for better sorting +CREATE INDEX IF NOT EXISTS idx_media_items_series_order ON media_items(series, series_number); +CREATE INDEX IF NOT EXISTS idx_media_items_date_published ON media_items(date_published); +``` + +#### Rationale for Each Field: +- **language**: Multi-lingual collections, filter by language +- **edition**: "2nd Edition", "Revised", "Collector's Edition" +- **page_count**: Display in UI, progress calculation, sorting +- **goodreads_id/openlibrary_id/google_books_id**: External service integration, better metadata fetching +- **copyright_year**: Original publication date (distinct from date_published reprints) +- **genre**: Structured category vs freeform tags +- **subjects**: Array of subjects (["Science Fiction", "Space Opera"]) + +#### Files to Modify: +1. `database/schema/schema.sql` - Add columns and indexes +2. Regenerate with `sqlc generate` (updates `internal/database/`) +3. Update scanner/metadata extraction to populate new fields +4. Update `internal/handlers/ebook.go` - UpdateMediaItem handler +5. Update UI templates to display new fields + +--- + +### **PHASE 2: Fix Sorting Functionality** โฌ…๏ธ **DO THIS SECOND** + +#### Problem Identified: +- **Location**: `templates/dashboard.templ:102-107, 222` +- **Issue**: Sort dropdown sends `sort` parameter but backend ignores it +- **Current**: Hardcoded `ORDER BY mi.created_at DESC` +- **Files**: `internal/handlers/ebook.go:824-865`, `internal/database/queries/queries.sql:107-121` + +#### Solution - Dynamic Sorting SQL: + +```sql +-- Add to internal/database/queries/queries.sql + +-- name: ListMediaItemsSorted :many +SELECT mi.*, l.name as library_name, lt.name as library_type_name +FROM media_items mi +JOIN libraries l ON mi.library_id = l.id +JOIN library_types lt ON l.library_type_id = lt.id +WHERE mi.library_id = $1 +ORDER BY + CASE + WHEN $2 = 'title ASC' THEN mi.title + WHEN $2 = 'author ASC' THEN COALESCE(mi.author, '') + WHEN $2 = 'series ASC' THEN COALESCE(mi.series, '') + WHEN $2 = 'date_published DESC' THEN mi.date_published::text + WHEN $2 = 'copyright_year DESC' THEN COALESCE(mi.copyright_year::text, '') + WHEN $2 = 'page_count ASC' THEN COALESCE(mi.page_count::text, '0') + WHEN $2 = 'genre ASC' THEN COALESCE(mi.genre, '') + ELSE mi.created_at::text + END + CASE WHEN $2 LIKE '%DESC' THEN DESC ELSE ASC END, + mi.title ASC -- Secondary sort for consistency +LIMIT $3 OFFSET $4; +``` + +#### Backend Handler Update: +```go +// internal/handlers/ebook.go - ListMediaItems function + +func (h *Handler) ListMediaItems(c echo.Context) error { + libraryID := c.QueryParam("library_id") + sort := c.QueryParam("sort") // <-- Process this parameter + limit, _ := strconv.Atoi(c.QueryParam("limit")) + offset, _ := strconv.Atoi(c.QueryParam("offset")) + + if limit == 0 { + limit = 50 + } + if limit > maxPaginationLimit { + limit = maxPaginationLimit + } + + // Default sort + if sort == "" { + sort = "created_at DESC" + } + + libUUID, err := uuid.Parse(libraryID) + if err != nil { + return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid library id"}) + } + + // Use new sorted query + items, err := h.db.ListMediaItemsSorted(c.Request().Context(), database.ListMediaItemsSortedParams{ + LibraryID: pgtype.UUID{Bytes: libUUID, Valid: true}, + Sort: pgtype.Text{String: sort, Valid: true}, + Limit: pgtype.Int4{Int32: int32(limit), Valid: true}, + Offset: pgtype.Int4{Int32: int32(offset), Valid: true}, + }) + + if err != nil { + return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()}) + } + + return c.JSON(http.StatusOK, map[string]interface{}{"data": items}) +} +``` + +#### Update Sort Dropdown in Templates: +```html + + +``` + +--- + +### **PHASE 3: Server-Side Filtering** โฌ…๏ธ **DO THIS THIRD** + +#### Current Problem: +- Client-side filtering only (`templates/dashboard.templ:371-383`) +- Breaks with pagination +- No filter persistence + +#### Solution - New Filtered Query: + +```sql +-- internal/database/queries/queries.sql + +-- name: ListMediaItemsFiltered :many +SELECT mi.*, l.name as library_name, lt.name as library_type_name +FROM media_items mi +JOIN libraries l ON mi.library_id = l.id +JOIN library_types lt ON l.library_type_id = lt.id +LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = $2 +WHERE mi.library_id = $1 + AND COALESCE(lv.is_visible, true) = true + AND ($3 = '' OR mi.author ILIKE $3) -- author_filter + AND ($4 = '' OR mi.series ILIKE $4) -- series_filter + AND ($5 = '' OR mi.genre = $5) -- genre_filter + AND ($6 = 0 OR mi.copyright_year >= $6) -- year_min + AND ($7 = 0 OR mi.copyright_year <= $7) -- year_max + AND ($8 = false OR mi.cover_image_path IS NOT NULL) -- has_cover +ORDER BY + CASE + WHEN $9 = 'title ASC' THEN mi.title + WHEN $9 = 'author ASC' THEN COALESCE(mi.author, '') + ELSE mi.created_at::text + END + CASE WHEN $9 LIKE '%DESC' THEN DESC ELSE ASC END, + mi.title ASC +LIMIT $10 OFFSET $11; +``` + +#### Filter UI to Add: +```html + +
+
+ Filters +
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+ +
+ + +
+
+
+ +
+
+
+ + +
+
+
+ + + +``` + +#### JavaScript for Filter Management: +```javascript +// Add to templates/dashboard.templ script section + +function applyFilters() { + const filters = { + author: document.getElementById('filter-author').value, + genre: document.getElementById('filter-genre').value, + series: document.getElementById('filter-series').value, + language: document.getElementById('filter-language').value, + yearMin: parseInt(document.getElementById('filter-year-min').value) || 0, + yearMax: parseInt(document.getElementById('filter-year-max').value) || 0, + hasCover: document.getElementById('filter-has-cover').checked + }; + + const sort = document.getElementById('sort-select').value; + + // Update URL for shareability + const params = new URLSearchParams(); + params.set('library_id', currentLibrary.id); + params.set('sort', sort); + Object.entries(filters).forEach(([key, value]) => { + if (value) params.set(`filters[${key}]`, value); + }); + + window.history.pushState({}, '', `?${params.toString()}`); + + loadMediaItemsWithFilters(filters, sort); + displayActiveFilters(filters); +} + +function loadMediaItemsWithFilters(filters, sort) { + const params = new URLSearchParams(); + params.set('library_id', currentLibrary.id); + params.set('sort', sort); + params.set('author_filter', filters.author); + params.set('genre_filter', filters.genre); + params.set('series_filter', filters.series); + params.set('year_min', filters.yearMin); + params.set('year_max', filters.yearMax); + params.set('has_cover', filters.hasCover); + + fetch(`/api/media-items/filtered?${params.toString()}`, { + headers: { + 'Authorization': 'Bearer ' + localStorage.getItem('token'), + 'Content-Type': 'application/json' + } + }) + .then(response => response.json()) + .then(data => { + mediaItems = data.data || data; + renderMediaItems(); + }); +} + +function displayActiveFilters(filters) { + const container = document.getElementById('active-filters'); + const chips = container.querySelector('.flex'); + chips.innerHTML = ''; + + let hasFilters = false; + Object.entries(filters).forEach(([key, value]) => { + if (value && value !== 0) { + hasFilters = true; + const chip = document.createElement('span'); + chip.className = 'px-3 py-1 rounded-full text-sm flex items-center gap-2'; + chip.style.cssText = 'background-color: var(--accent); color: var(--bg-primary)'; + chip.innerHTML = `${key}: ${value} `; + chips.appendChild(chip); + } + }); + + container.classList.toggle('hidden', !hasFilters); +} + +function clearFilters() { + document.getElementById('filter-author').value = ''; + document.getElementById('filter-genre').value = ''; + document.getElementById('filter-series').value = ''; + document.getElementById('filter-language').value = ''; + document.getElementById('filter-year-min').value = ''; + document.getElementById('filter-year-max').value = ''; + document.getElementById('filter-has-cover').checked = false; + + applyFilters(); +} + +// Load filters from URL on page load +document.addEventListener('DOMContentLoaded', function() { + const params = new URLSearchParams(window.location.search); + if (params.has('filters[author]')) { + document.getElementById('filter-author').value = params.get('filters[author]'); + } + // ... load other filters + + if (params.has('sort')) { + document.getElementById('sort-select').value = params.get('sort'); + } +}); +``` + +--- + +## ๐Ÿ“ KEY FILES & LOCATIONS + +### Database: +- `database/schema/schema.sql` - Main schema (merge changes here) +- `internal/database/queries/queries.sql` - SQL queries (add new ones) +- `internal/database/queries.sql.go` - Auto-generated by sqlc + +### Handlers: +- `internal/handlers/ebook.go:824-865` - ListMediaItems endpoint +- `internal/handlers/ebook.go:1673-1728` - SearchMediaItems (working reference) + +### Templates: +- `templates/dashboard.templ:102-107` - Sort dropdown UI +- `templates/dashboard.templ:222` - API call with sort param +- `templates/dashboard.templ:371-383` - Client-side filter (needs replacement) +- `templates/bookshelf.templ` - Bookshelf view +- `templates/header.templ` - Search box + +### Tests: +- `cmd/server/tests/search_test.go` - Reference for test structure +- `bruno/media-items/Search Media Items.bru` - API documentation + +### Static Files: +- `web/static/search.js` - Search implementation (working reference) +- `web/static/header.js` - Header functionality + +--- + +## ๐Ÿ”„ IMPLEMENTATION CHECKLIST + +### Phase 1: Add Database Fields +- [ ] Update `database/schema/schema.sql` with new columns +- [ ] Add indexes for new columns +- [ ] Delete current database +- [ ] Rebuild with clean Podman cache: `podman-compose down -v && podman-compose up --build` +- [ ] Run `sqlc generate` to update Go code +- [ ] Update scanner handlers to populate new fields +- [ ] Test with sample data + +### Phase 2: Fix Sorting +- [ ] Add `ListMediaItemsSorted` query to `queries.sql` +- [ ] Regenerate with `sqlc generate` +- [ ] Update `ListMediaItems` handler in `ebook.go` +- [ ] Update sort dropdown options in `dashboard.templ` +- [ ] Create Bruno test for sorting +- [ ] Write tests for no user, user, admin contexts +- [ ] Update README.md if needed + +### Phase 3: Add Filtering +- [ ] Add `ListMediaItemsFiltered` query to `queries.sql` +- [ ] Create handler endpoint for filtered results +- [ ] Add filter UI to `dashboard.templ` +- [ ] Implement JavaScript filter logic +- [ ] Add URL state management +- [ ] Create Bruno tests for filters +- [ ] Test all filter combinations +- [ ] Update README.md + +--- + +## ๐Ÿงช TESTING REQUIREMENTS + +Each phase must include tests covering: +1. **No user context** (unauthenticated requests) +2. **User context** (regular user with library visibility) +3. **Admin context** (admin with full access) + +Example test structure (from `search_test.go`): +```go +t.Run("User context - sorted results", func(t *testing.T) { + req := httptest.NewRequest("GET", "/api/media-items?library_id=xxx&sort=title+ASC", nil) + req.Header.Set("Authorization", "Bearer valid-user-token") + // ... assertions +}) +``` + +--- + +## ๐Ÿ“Š CURRENT STATE REFERENCE + +### Working Search API: +``` +GET /api/media-items/search?q=harry +Response: Array of media items with highlighted matches +Status: 200 (results), 404 (no results), 401 (unauthorized) +``` + +### Broken Sort API: +``` +GET /api/media-items?library_id=xxx&sort=title+ASC +Current behavior: Ignores sort parameter, returns default order +Expected behavior: Returns results sorted by title ASC +``` + +### Existing Schema (23 fields): +id, library_id, title, author, isbn, description, file_path, file_size, mime_type, cover_image_path, series, series_number, tags, asin, date_published, publisher, contributors, added_by_admin_id, created_at, updated_at + +### Recommended New Fields (9 fields): +language, edition, page_count, goodreads_id, openlibrary_id, google_books_id, copyright_year, genre, subjects + +--- + +## โš ๏ธ IMPORTANT NOTES + +1. **DO NOT create new migration files** - merge into existing schema.sql +2. **Delete and rebuild database** when schema changes (per your standards) +3. **Use Podman** for all container operations +4. **Include Bruno tests** for all API changes +5. **Follow pgx v5 patterns** throughout +6. **Multiple git commits** preferred (one logical change per commit) +7. **Test all three contexts** (no user, user, admin) +8. **Preserve existing functionality** - search is working, don't break it + +--- + +## ๐Ÿš€ QUICK START COMMANDS + +```bash +# After schema changes, rebuild database +podman-compose down -v +podman-compose up --build + +# Regenerate Go database code +cd internal/database && sqlc generate + +# Run tests +go test ./cmd/server/tests/... + +# Access application +# URL: http://localhost:8765 +``` + +--- + +## ๐Ÿ“ NEXT STEPS (When Resuming) + +1. Start with **Phase 1** (add database fields) +2. Follow **PROJECT STANDARDS** strictly +3. Create git commits after each working phase +4. Update this summary as you progress +5. Run tests frequently to catch regressions early + +--- + +**End of Summary** diff --git a/README.md b/README.md index a348659..75bcab1 100644 --- a/README.md +++ b/README.md @@ -44,13 +44,38 @@ A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and T - **Universal Media Support**: Single system for all media types with unified interface - **Rich Metadata**: Automatic extraction of title, author, series, publisher, ISBN, tags - **ISBN Normalization**: Automatic ISBN format normalization (removes hyphens and spaces) supporting ISBN-10 and ISBN-13 formats - - **Advanced Search**: - - Partial matching search across title, author, series, tags, and contributors - - Automatic fuzzy search fallback when no partial matches found (handles typos and misspellings) - - Real-time search results with highlighted matches - - Keyboard navigation (โ†‘โ†“ arrows, Enter to select, Escape to close) - - Respects library visibility settings - - **Advanced Rating**: 5-star system with half-star precision (1-10 scale) + - **Advanced Search**: + - Partial matching search across title, author, series, tags, and contributors + - Automatic fuzzy search fallback when no partial matches found (handles typos and misspellings) + - Real-time search results with highlighted matches + - Keyboard navigation (โ†‘โ†“ arrows, Enter to select, Escape to close) + - Respects library visibility settings + - **Dynamic Sorting**: Sort your media collection by multiple fields + - Title (A-Z or Z-A) + - Author (A-Z or Z-A) + - Date added (newest or oldest first) + - Date published (newest or oldest first) + - Copyright year (newest or oldest first) + - Series order (with series number) + - Page count (shortest or longest first) + - Genre (A-Z or Z-A) + - **Advanced Filtering**: Filter media items with multiple options + - Filter by author (partial match) + - Filter by series (partial match) + - Filter by genre (exact match) + - Filter by language (English, Spanish, French, German, Japanese, Chinese, Korean, Russian, Italian, Portuguese) + - Filter by copyright year range + - Filter by items with cover images only + - Combine multiple filters with URL state management for shareable links + - **Enhanced Metadata**: New fields for better organization + - Language support for multi-lingual collections + - Edition information (2nd Edition, Revised, Collector's Edition, etc.) + - Page count for better sorting and progress calculation + - External service integration (Goodreads, OpenLibrary, Google Books IDs) + - Copyright year (distinct from publication date) + - Structured genre classification + - Subject tags (array of subjects) + - **Advanced Rating**: 5-star system with half-star precision (1-10 scale) - **Reading Progress**: User-specific progress tracking with current page and total pages - **Notes & Highlights**: Personal annotations and text highlighting with color customization - **Highlight Notes**: Link highlights to detailed notes for comprehensive annotations @@ -61,13 +86,13 @@ A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and T ## ๐Ÿš€ Quick Start ### Prerequisites -- Docker and Docker Compose -- PostgreSQL database (handled by Docker) +- **Podman** and Podman Compose (recommended) or Docker +- PostgreSQL database (handled by Podman) ### Environment Setup 1. **Create .env file**: ```bash -cp .env.example .env +cp .env.example ``` 2. **Edit .env** with your secure values: @@ -78,13 +103,16 @@ DBPASS="your-secure-database-password-here" ### Running the Application ```bash -# Option 1: Using .env file +# Using Podman (recommended) +podman-compose up --build + +# Using Docker docker-compose up --build # Option 2: Direct environment variables export JWT_SECRET="your-secure-jwt-secret-key-here" export DBPASS="your-secure-database-password-here" -docker-compose up --build +podman-compose up --build ``` Access the application at: **http://localhost:8765**