bookhoard/IMPLEMENTATION_SUMMARY.md

# 📚 Bookmann Project Conversation Summary

**Date**: January 30, 2026
**Repository**: `/home/nymusicman/Code/bookmann`
**Latest Commit**: `1411c2a`
**Working Directory**: `/home/nymusicman/Code/bookmann`

---

## 🎯 Project Overview

Bookmann is a **self-hosted media library system** supporting ebooks, comics, and manga with:
- **Backend**: Go 1.25+, pgx v5, PostgreSQL 15+, Echo framework
- **Frontend**: HTMX, Tailwind CSS, Vanilla JavaScript
- **Architecture**: Multi-library system with user/admin roles
- **Current Status**: Search fully implemented, sorting/filtering partially implemented

---

## ⚙️ PROJECT STANDARDS (Must Follow)

1. **Database Changes**: pgx v5 standards only
2. **Container Runtime**: Use Podman (NOT Docker)
3. **Build System**: All builds through existing Dockerfile/docker-compose
4. **Migrations**: NO new migration files - merge changes into current one until release
5. **API Changes**: Include Bruno requests with documentation
6. **Tests**: Must cover no user, user, and admin contexts
7. **Project Structure**: Minimize changes, place files appropriately
8. **Documentation**: Update README.md when users/admins need to be informed
9. **Environment**: Auto-generate .env if missing
10. **Functionality**: Never break existing features unless explicitly instructed
11. **Git Workflow**: Prefer multiple organized commits
12. **Database Tables**: When schema changes, delete database and rebuild with clean Podman cache

---

## 📋 IMPLEMENTATION PLAN (Reordered Priority)

### **PHASE 1: Add Missing Fields to media_items Table** ⬅️ **DO THIS FIRST**

#### Recommended Fields to Add (Priority 1 + 2)

```sql
-- High-priority fields for better filtering and metadata
ALTER TABLE media_items
    ADD COLUMN language VARCHAR(10) DEFAULT 'en',
    ADD COLUMN edition VARCHAR(255),
    ADD COLUMN page_count INTEGER,
    ADD COLUMN goodreads_id VARCHAR(20),
    ADD COLUMN openlibrary_id VARCHAR(100),
    ADD COLUMN google_books_id VARCHAR(100),
    ADD COLUMN copyright_year INTEGER,
    ADD COLUMN genre VARCHAR(100),
    ADD COLUMN subjects TEXT[];

-- Add indexes for new fields
CREATE INDEX idx_media_items_language ON media_items(language);
CREATE INDEX idx_media_items_genre ON media_items(genre);
CREATE INDEX idx_media_items_page_count ON media_items(page_count);
CREATE INDEX idx_media_items_copyright_year ON media_items(copyright_year);

-- Update existing indexes for better sorting
CREATE INDEX IF NOT EXISTS idx_media_items_series_order ON media_items(series, series_number);
CREATE INDEX IF NOT EXISTS idx_media_items_date_published ON media_items(date_published);
```

#### Rationale for Each Field:
- **language**: Multi-lingual collections, filter by language
- **edition**: "2nd Edition", "Revised", "Collector's Edition"
- **page_count**: Display in UI, progress calculation, sorting
- **goodreads_id/openlibrary_id/google_books_id**: External service integration, better metadata fetching
- **copyright_year**: Original publication date (distinct from date_published reprints)
- **genre**: Structured category vs freeform tags
- **subjects**: Array of subjects (["Science Fiction", "Space Opera"])

#### Files to Modify:
1. `database/schema/schema.sql` - Add columns and indexes
2. Regenerate with `sqlc generate` (updates `internal/database/`)
3. Update scanner/metadata extraction to populate new fields
4. Update `internal/handlers/ebook.go` - UpdateMediaItem handler
5. Update UI templates to display new fields

---

### **PHASE 2: Fix Sorting Functionality** ⬅️ **DO THIS SECOND**

#### Problem Identified:
- **Location**: `templates/dashboard.templ:102-107, 222`
- **Issue**: Sort dropdown sends `sort` parameter but backend ignores it
- **Current**: Hardcoded `ORDER BY mi.created_at DESC`
- **Files**: `internal/handlers/ebook.go:824-865`, `internal/database/queries/queries.sql:107-121`

#### Solution - Dynamic Sorting SQL:

```sql
-- Add to internal/database/queries/queries.sql

-- name: ListMediaItemsSorted :many
SELECT mi.*, l.name as library_name, lt.name as library_type_name
FROM media_items mi
JOIN libraries l ON mi.library_id = l.id
JOIN library_types lt ON l.library_type_id = lt.id
WHERE mi.library_id = $1
ORDER BY
    CASE
        WHEN $2 = 'title ASC' THEN mi.title
        WHEN $2 = 'author ASC' THEN COALESCE(mi.author, '')
        WHEN $2 = 'series ASC' THEN COALESCE(mi.series, '')
        WHEN $2 = 'date_published DESC' THEN mi.date_published::text
        WHEN $2 = 'copyright_year DESC' THEN COALESCE(mi.copyright_year::text, '')
        WHEN $2 = 'page_count ASC' THEN COALESCE(mi.page_count::text, '0')
        WHEN $2 = 'genre ASC' THEN COALESCE(mi.genre, '')
        ELSE mi.created_at::text
    END
    CASE WHEN $2 LIKE '%DESC' THEN DESC ELSE ASC END,
    mi.title ASC -- Secondary sort for consistency
LIMIT $3 OFFSET $4;
```

#### Backend Handler Update:
```go
// internal/handlers/ebook.go - ListMediaItems function

func (h *Handler) ListMediaItems(c echo.Context) error {
    libraryID := c.QueryParam("library_id")
    sort := c.QueryParam("sort") // <-- Process this parameter
    limit, _ := strconv.Atoi(c.QueryParam("limit"))
    offset, _ := strconv.Atoi(c.QueryParam("offset"))

    if limit == 0 {
        limit = 50
    }
    if limit > maxPaginationLimit {
        limit = maxPaginationLimit
    }

    // Default sort
    if sort == "" {
        sort = "created_at DESC"
    }

    libUUID, err := uuid.Parse(libraryID)
    if err != nil {
        return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid library id"})
    }

    // Use new sorted query
    items, err := h.db.ListMediaItemsSorted(c.Request().Context(), database.ListMediaItemsSortedParams{
        LibraryID: pgtype.UUID{Bytes: libUUID, Valid: true},
        Sort:      pgtype.Text{String: sort, Valid: true},
        Limit:     pgtype.Int4{Int32: int32(limit), Valid: true},
        Offset:    pgtype.Int4{Int32: int32(offset), Valid: true},
    })

    if err != nil {
        return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
    }

    return c.JSON(http.StatusOK, map[string]interface{}{"data": items})
}
```

#### Update Sort Dropdown in Templates:
```html
<!-- templates/dashboard.templ -->
<select id="sort-select" onchange="loadMediaItems()">
    <option value="created_at DESC">Newest First</option>
    <option value="created_at ASC">Oldest First</option>
    <option value="title ASC">Title A-Z</option>
    <option value="title DESC">Title Z-A</option>
    <option value="author ASC">Author A-Z</option>
    <option value="author DESC">Author Z-A</option>
    <option value="series ASC, series_number ASC">Series Order</option>
    <option value="date_published DESC">Newest Published</option>
    <option value="copyright_year DESC">Copyright Year</option>
    <option value="page_count ASC">Shortest First</option>
    <option value="page_count DESC">Longest First</option>
    <option value="genre ASC">Genre A-Z</option>
</select>
```

---

### **PHASE 3: Server-Side Filtering** ⬅️ **DO THIS THIRD**

#### Current Problem:
- Client-side filtering only (`templates/dashboard.templ:371-383`)
- Breaks with pagination
- No filter persistence

#### Solution - New Filtered Query:

```sql
-- internal/database/queries/queries.sql

-- name: ListMediaItemsFiltered :many
SELECT mi.*, l.name as library_name, lt.name as library_type_name
FROM media_items mi
JOIN libraries l ON mi.library_id = l.id
JOIN library_types lt ON l.library_type_id = lt.id
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = $2
WHERE mi.library_id = $1
  AND COALESCE(lv.is_visible, true) = true
  AND ($3 = '' OR mi.author ILIKE $3) -- author_filter
  AND ($4 = '' OR mi.series ILIKE $4) -- series_filter
  AND ($5 = '' OR mi.genre = $5) -- genre_filter
  AND ($6 = 0 OR mi.copyright_year >= $6) -- year_min
  AND ($7 = 0 OR mi.copyright_year <= $7) -- year_max
  AND ($8 = false OR mi.cover_image_path IS NOT NULL) -- has_cover
ORDER BY
    CASE
        WHEN $9 = 'title ASC' THEN mi.title
        WHEN $9 = 'author ASC' THEN COALESCE(mi.author, '')
        ELSE mi.created_at::text
    END
    CASE WHEN $9 LIKE '%DESC' THEN DESC ELSE ASC END,
    mi.title ASC
LIMIT $10 OFFSET $11;
```

#### Filter UI to Add:
```html
<!-- Add above media grid in templates/dashboard.templ -->
<div class="filter-panel mb-6">
    <details>
        <summary class="cursor-pointer font-semibold mb-4">Filters</summary>
        <div class="grid grid-cols-2 md:grid-cols-4 gap-4">
            <div>
                <label class="block text-sm mb-1">Author</label>
                <select id="filter-author" class="w-full px-3 py-2 border rounded">
                    <option value="">All Authors</option>
                    <!-- Populated dynamically -->
                </select>
            </div>
            <div>
                <label class="block text-sm mb-1">Genre</label>
                <select id="filter-genre" class="w-full px-3 py-2 border rounded">
                    <option value="">All Genres</option>
                    <!-- Populated from distinct genres -->
                </select>
            </div>
            <div>
                <label class="block text-sm mb-1">Series</label>
                <select id="filter-series" class="w-full px-3 py-2 border rounded">
                    <option value="">All Series</option>
                    <!-- Populated dynamically -->
                </select>
            </div>
            <div>
                <label class="block text-sm mb-1">Language</label>
                <select id="filter-language" class="w-full px-3 py-2 border rounded">
                    <option value="">All Languages</option>
                    <option value="en">English</option>
                    <option value="es">Spanish</option>
                    <option value="fr">French</option>
                    <!-- Add more as needed -->
                </select>
            </div>
            <div>
                <label class="block text-sm mb-1">Year Range</label>
                <div class="flex gap-2">
                    <input type="number" id="filter-year-min" placeholder="From" class="w-1/2 px-3 py-2 border rounded" min="1800" max="2100">
                    <input type="number" id="filter-year-max" placeholder="To" class="w-1/2 px-3 py-2 border rounded" min="1800" max="2100">
                </div>
            </div>
            <div class="flex items-end">
                <label class="flex items-center">
                    <input type="checkbox" id="filter-has-cover" class="mr-2">
                    <span class="text-sm">Has Cover</span>
                </label>
            </div>
        </div>
        <div class="mt-4">
            <button onclick="applyFilters()" class="btn-primary px-4 py-2 rounded">Apply Filters</button>
            <button onclick="clearFilters()" class="btn-secondary px-4 py-2 rounded ml-2">Clear All</button>
        </div>
    </details>
</div>

<!-- Active filters display -->
<div id="active-filters" class="hidden mb-4">
    <div class="flex flex-wrap gap-2">
        <!-- Active filter chips added here dynamically -->
    </div>
</div>
```

#### JavaScript for Filter Management:
```javascript
// Add to templates/dashboard.templ script section

function applyFilters() {
    const filters = {
        author: document.getElementById('filter-author').value,
        genre: document.getElementById('filter-genre').value,
        series: document.getElementById('filter-series').value,
        language: document.getElementById('filter-language').value,
        yearMin: parseInt(document.getElementById('filter-year-min').value) || 0,
        yearMax: parseInt(document.getElementById('filter-year-max').value) || 0,
        hasCover: document.getElementById('filter-has-cover').checked
    };

    const sort = document.getElementById('sort-select').value;

    // Update URL for shareability
    const params = new URLSearchParams();
    params.set('library_id', currentLibrary.id);
    params.set('sort', sort);
    Object.entries(filters).forEach(([key, value]) => {
        if (value) params.set(`filters[${key}]`, value);
    });

    window.history.pushState({}, '', `?${params.toString()}`);

    loadMediaItemsWithFilters(filters, sort);
    displayActiveFilters(filters);
}

function loadMediaItemsWithFilters(filters, sort) {
    const params = new URLSearchParams();
    params.set('library_id', currentLibrary.id);
    params.set('sort', sort);
    params.set('author_filter', filters.author);
    params.set('genre_filter', filters.genre);
    params.set('series_filter', filters.series);
    params.set('year_min', filters.yearMin);
    params.set('year_max', filters.yearMax);
    params.set('has_cover', filters.hasCover);

    fetch(`/api/media-items/filtered?${params.toString()}`, {
        headers: {
            'Authorization': 'Bearer ' + localStorage.getItem('token'),
            'Content-Type': 'application/json'
        }
    })
    .then(response => response.json())
    .then(data => {
        mediaItems = data.data || data;
        renderMediaItems();
    });
}

function displayActiveFilters(filters) {
    const container = document.getElementById('active-filters');
    const chips = container.querySelector('.flex');
    chips.innerHTML = '';

    let hasFilters = false;
    Object.entries(filters).forEach(([key, value]) => {
        if (value && value !== 0) {
            hasFilters = true;
            const chip = document.createElement('span');
            chip.className = 'px-3 py-1 rounded-full text-sm flex items-center gap-2';
            chip.style.cssText = 'background-color: var(--accent); color: var(--bg-primary)';
            chip.innerHTML = `${key}: ${value} <button onclick="clearFilter('${key}')" class="font-bold">×</button>`;
            chips.appendChild(chip);
        }
    });

    container.classList.toggle('hidden', !hasFilters);
}

function clearFilters() {
    document.getElementById('filter-author').value = '';
    document.getElementById('filter-genre').value = '';
    document.getElementById('filter-series').value = '';
    document.getElementById('filter-language').value = '';
    document.getElementById('filter-year-min').value = '';
    document.getElementById('filter-year-max').value = '';
    document.getElementById('filter-has-cover').checked = false;

    applyFilters();
}

// Load filters from URL on page load
document.addEventListener('DOMContentLoaded', function() {
    const params = new URLSearchParams(window.location.search);
    if (params.has('filters[author]')) {
        document.getElementById('filter-author').value = params.get('filters[author]');
    }
    // ... load other filters

    if (params.has('sort')) {
        document.getElementById('sort-select').value = params.get('sort');
    }
});
```

---

## 📁 KEY FILES & LOCATIONS

### Database:
- `database/schema/schema.sql` - Main schema (merge changes here)
- `internal/database/queries/queries.sql` - SQL queries (add new ones)
- `internal/database/queries.sql.go` - Auto-generated by sqlc

### Handlers:
- `internal/handlers/ebook.go:824-865` - ListMediaItems endpoint
- `internal/handlers/ebook.go:1673-1728` - SearchMediaItems (working reference)

### Templates:
- `templates/dashboard.templ:102-107` - Sort dropdown UI
- `templates/dashboard.templ:222` - API call with sort param
- `templates/dashboard.templ:371-383` - Client-side filter (needs replacement)
- `templates/bookshelf.templ` - Bookshelf view
- `templates/header.templ` - Search box

### Tests:
- `cmd/server/tests/search_test.go` - Reference for test structure
- `bruno/media-items/Search Media Items.bru` - API documentation

### Static Files:
- `web/static/search.js` - Search implementation (working reference)
- `web/static/header.js` - Header functionality

---

## 🔄 IMPLEMENTATION CHECKLIST

### Phase 1: Add Database Fields
- [ ] Update `database/schema/schema.sql` with new columns
- [ ] Add indexes for new columns
- [ ] Delete current database
- [ ] Rebuild with clean Podman cache: `podman-compose down -v && podman-compose up --build`
- [ ] Run `sqlc generate` to update Go code
- [ ] Update scanner handlers to populate new fields
- [ ] Test with sample data

### Phase 2: Fix Sorting
- [ ] Add `ListMediaItemsSorted` query to `queries.sql`
- [ ] Regenerate with `sqlc generate`
- [ ] Update `ListMediaItems` handler in `ebook.go`
- [ ] Update sort dropdown options in `dashboard.templ`
- [ ] Create Bruno test for sorting
- [ ] Write tests for no user, user, admin contexts
- [ ] Update README.md if needed

### Phase 3: Add Filtering
- [ ] Add `ListMediaItemsFiltered` query to `queries.sql`
- [ ] Create handler endpoint for filtered results
- [ ] Add filter UI to `dashboard.templ`
- [ ] Implement JavaScript filter logic
- [ ] Add URL state management
- [ ] Create Bruno tests for filters
- [ ] Test all filter combinations
- [ ] Update README.md

---

## 🧪 TESTING REQUIREMENTS

Each phase must include tests covering:
1. **No user context** (unauthenticated requests)
2. **User context** (regular user with library visibility)
3. **Admin context** (admin with full access)

Example test structure (from `search_test.go`):
```go
t.Run("User context - sorted results", func(t *testing.T) {
    req := httptest.NewRequest("GET", "/api/media-items?library_id=xxx&sort=title+ASC", nil)
    req.Header.Set("Authorization", "Bearer valid-user-token")
    // ... assertions
})
```

---

## 📊 CURRENT STATE REFERENCE

### Working Search API:
```
GET /api/media-items/search?q=harry
Response: Array of media items with highlighted matches
Status: 200 (results), 404 (no results), 401 (unauthorized)
```

### Broken Sort API:
```
GET /api/media-items?library_id=xxx&sort=title+ASC
Current behavior: Ignores sort parameter, returns default order
Expected behavior: Returns results sorted by title ASC
```

### Existing Schema (23 fields):
id, library_id, title, author, isbn, description, file_path, file_size, mime_type, cover_image_path, series, series_number, tags, asin, date_published, publisher, contributors, added_by_admin_id, created_at, updated_at

### Recommended New Fields (9 fields):
language, edition, page_count, goodreads_id, openlibrary_id, google_books_id, copyright_year, genre, subjects

---

## ⚠️ IMPORTANT NOTES

1. **DO NOT create new migration files** - merge into existing schema.sql
2. **Delete and rebuild database** when schema changes (per your standards)
3. **Use Podman** for all container operations
4. **Include Bruno tests** for all API changes
5. **Follow pgx v5 patterns** throughout
6. **Multiple git commits** preferred (one logical change per commit)
7. **Test all three contexts** (no user, user, admin)
8. **Preserve existing functionality** - search is working, don't break it

---

## 🚀 QUICK START COMMANDS

```bash
# After schema changes, rebuild database
podman-compose down -v
podman-compose up --build

# Regenerate Go database code
cd internal/database && sqlc generate

# Run tests
go test ./cmd/server/tests/...

# Access application
# URL: http://localhost:8765
```

---

## 📝 NEXT STEPS (When Resuming)

1. Start with **Phase 1** (add database fields)
2. Follow **PROJECT STANDARDS** strictly
3. Create git commits after each working phase
4. Update this summary as you progress
5. Run tests frequently to catch regressions early

---

**End of Summary**