bookhoard/IMPLEMENTATION_SUMMARY.md

📚 Bookmann Project Conversation Summary

Date: January 30, 2026 Repository: /home/nymusicman/Code/bookmann Latest Commit: 1411c2a Working Directory: /home/nymusicman/Code/bookmann

---

🎯 Project Overview

Bookmann is a self-hosted media library system supporting ebooks, comics, and manga with:

• Backend: Go 1.25+, pgx v5, PostgreSQL 15+, Echo framework
• Frontend: HTMX, Tailwind CSS, Vanilla JavaScript
• Architecture: Multi-library system with user/admin roles
• Current Status: Search fully implemented, sorting/filtering partially implemented

---

⚙️ PROJECT STANDARDS (Must Follow)

1. Database Changes: pgx v5 standards only
2. Container Runtime: Use Podman (NOT Docker)
3. Build System: All builds through existing Dockerfile/docker-compose
4. Migrations: NO new migration files - merge changes into current one until release
5. API Changes: Include Bruno requests with documentation
6. Tests: Must cover no user, user, and admin contexts
7. Project Structure: Minimize changes, place files appropriately
8. Documentation: Update README.md when users/admins need to be informed
9. Environment: Auto-generate .env if missing
10. Functionality: Never break existing features unless explicitly instructed
11. Git Workflow: Prefer multiple organized commits
12. Database Tables: When schema changes, delete database and rebuild with clean Podman cache

---

📋 IMPLEMENTATION PLAN (Reordered Priority)

PHASE 1: Add Missing Fields to media_items Table ⬅️ DO THIS FIRST

Recommended Fields to Add (Priority 1 + 2)

-- High-priority fields for better filtering and metadata
ALTER TABLE media_items
    ADD COLUMN language VARCHAR(10) DEFAULT 'en',
    ADD COLUMN edition VARCHAR(255),
    ADD COLUMN page_count INTEGER,
    ADD COLUMN goodreads_id VARCHAR(20),
    ADD COLUMN openlibrary_id VARCHAR(100),
    ADD COLUMN google_books_id VARCHAR(100),
    ADD COLUMN copyright_year INTEGER,
    ADD COLUMN genre VARCHAR(100),
    ADD COLUMN subjects TEXT[];

-- Add indexes for new fields
CREATE INDEX idx_media_items_language ON media_items(language);
CREATE INDEX idx_media_items_genre ON media_items(genre);
CREATE INDEX idx_media_items_page_count ON media_items(page_count);
CREATE INDEX idx_media_items_copyright_year ON media_items(copyright_year);

-- Update existing indexes for better sorting
CREATE INDEX IF NOT EXISTS idx_media_items_series_order ON media_items(series, series_number);
CREATE INDEX IF NOT EXISTS idx_media_items_date_published ON media_items(date_published);

Rationale for Each Field:

• language: Multi-lingual collections, filter by language
• edition: "2nd Edition", "Revised", "Collector's Edition"
• page_count: Display in UI, progress calculation, sorting
• goodreads_id/openlibrary_id/google_books_id: External service integration, better metadata fetching
• copyright_year: Original publication date (distinct from date_published reprints)
• genre: Structured category vs freeform tags
• subjects: Array of subjects (["Science Fiction", "Space Opera"])

Files to Modify:

1. database/schema/schema.sql - Add columns and indexes
2. Regenerate with sqlc generate (updates internal/database/)
3. Update scanner/metadata extraction to populate new fields
4. Update internal/handlers/ebook.go - UpdateMediaItem handler
5. Update UI templates to display new fields

---

PHASE 2: Fix Sorting Functionality ⬅️ DO THIS SECOND

Problem Identified:

• Location: templates/dashboard.templ:102-107, 222
• Issue: Sort dropdown sends sort parameter but backend ignores it
• Current: Hardcoded ORDER BY mi.created_at DESC
• Files: internal/handlers/ebook.go:824-865, internal/database/queries/queries.sql:107-121

Solution - Dynamic Sorting SQL:

-- Add to internal/database/queries/queries.sql

-- name: ListMediaItemsSorted :many
SELECT mi.*, l.name as library_name, lt.name as library_type_name
FROM media_items mi
JOIN libraries l ON mi.library_id = l.id
JOIN library_types lt ON l.library_type_id = lt.id
WHERE mi.library_id = $1
ORDER BY
    CASE
        WHEN $2 = 'title ASC' THEN mi.title
        WHEN $2 = 'author ASC' THEN COALESCE(mi.author, '')
        WHEN $2 = 'series ASC' THEN COALESCE(mi.series, '')
        WHEN $2 = 'date_published DESC' THEN mi.date_published::text
        WHEN $2 = 'copyright_year DESC' THEN COALESCE(mi.copyright_year::text, '')
        WHEN $2 = 'page_count ASC' THEN COALESCE(mi.page_count::text, '0')
        WHEN $2 = 'genre ASC' THEN COALESCE(mi.genre, '')
        ELSE mi.created_at::text
    END
    CASE WHEN $2 LIKE '%DESC' THEN DESC ELSE ASC END,
    mi.title ASC -- Secondary sort for consistency
LIMIT $3 OFFSET $4;

Backend Handler Update:

// internal/handlers/ebook.go - ListMediaItems function

func (h *Handler) ListMediaItems(c echo.Context) error {
    libraryID := c.QueryParam("library_id")
    sort := c.QueryParam("sort") // <-- Process this parameter
    limit, _ := strconv.Atoi(c.QueryParam("limit"))
    offset, _ := strconv.Atoi(c.QueryParam("offset"))

    if limit == 0 {
        limit = 50
    }
    if limit > maxPaginationLimit {
        limit = maxPaginationLimit
    }

    // Default sort
    if sort == "" {
        sort = "created_at DESC"
    }

    libUUID, err := uuid.Parse(libraryID)
    if err != nil {
        return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid library id"})
    }

    // Use new sorted query
    items, err := h.db.ListMediaItemsSorted(c.Request().Context(), database.ListMediaItemsSortedParams{
        LibraryID: pgtype.UUID{Bytes: libUUID, Valid: true},
        Sort:      pgtype.Text{String: sort, Valid: true},
        Limit:     pgtype.Int4{Int32: int32(limit), Valid: true},
        Offset:    pgtype.Int4{Int32: int32(offset), Valid: true},
    })

    if err != nil {
        return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
    }

    return c.JSON(http.StatusOK, map[string]interface{}{"data": items})
}

Update Sort Dropdown in Templates:

<!-- templates/dashboard.templ -->
<select id="sort-select" onchange="loadMediaItems()">
    <option value="created_at DESC">Newest First</option>
    <option value="created_at ASC">Oldest First</option>
    <option value="title ASC">Title A-Z</option>
    <option value="title DESC">Title Z-A</option>
    <option value="author ASC">Author A-Z</option>
    <option value="author DESC">Author Z-A</option>
    <option value="series ASC, series_number ASC">Series Order</option>
    <option value="date_published DESC">Newest Published</option>
    <option value="copyright_year DESC">Copyright Year</option>
    <option value="page_count ASC">Shortest First</option>
    <option value="page_count DESC">Longest First</option>
    <option value="genre ASC">Genre A-Z</option>
</select>

---

PHASE 3: Server-Side Filtering ⬅️ DO THIS THIRD

Current Problem:

• Client-side filtering only (templates/dashboard.templ:371-383)
• Breaks with pagination
• No filter persistence

Solution - New Filtered Query:

-- internal/database/queries/queries.sql

-- name: ListMediaItemsFiltered :many
SELECT mi.*, l.name as library_name, lt.name as library_type_name
FROM media_items mi
JOIN libraries l ON mi.library_id = l.id
JOIN library_types lt ON l.library_type_id = lt.id
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = $2
WHERE mi.library_id = $1
  AND COALESCE(lv.is_visible, true) = true
  AND ($3 = '' OR mi.author ILIKE $3) -- author_filter
  AND ($4 = '' OR mi.series ILIKE $4) -- series_filter
  AND ($5 = '' OR mi.genre = $5) -- genre_filter
  AND ($6 = 0 OR mi.copyright_year >= $6) -- year_min
  AND ($7 = 0 OR mi.copyright_year <= $7) -- year_max
  AND ($8 = false OR mi.cover_image_path IS NOT NULL) -- has_cover
ORDER BY
    CASE
        WHEN $9 = 'title ASC' THEN mi.title
        WHEN $9 = 'author ASC' THEN COALESCE(mi.author, '')
        ELSE mi.created_at::text
    END
    CASE WHEN $9 LIKE '%DESC' THEN DESC ELSE ASC END,
    mi.title ASC
LIMIT $10 OFFSET $11;

Filter UI to Add:

<!-- Add above media grid in templates/dashboard.templ -->
<div class="filter-panel mb-6">
    <details>
        <summary class="cursor-pointer font-semibold mb-4">Filters</summary>
        <div class="grid grid-cols-2 md:grid-cols-4 gap-4">
            <div>
                <label class="block text-sm mb-1">Author</label>
                <select id="filter-author" class="w-full px-3 py-2 border rounded">
                    <option value="">All Authors</option>
                    <!-- Populated dynamically -->
                </select>
            </div>
            <div>
                <label class="block text-sm mb-1">Genre</label>
                <select id="filter-genre" class="w-full px-3 py-2 border rounded">
                    <option value="">All Genres</option>
                    <!-- Populated from distinct genres -->
                </select>
            </div>
            <div>
                <label class="block text-sm mb-1">Series</label>
                <select id="filter-series" class="w-full px-3 py-2 border rounded">
                    <option value="">All Series</option>
                    <!-- Populated dynamically -->
                </select>
            </div>
            <div>
                <label class="block text-sm mb-1">Language</label>
                <select id="filter-language" class="w-full px-3 py-2 border rounded">
                    <option value="">All Languages</option>
                    <option value="en">English</option>
                    <option value="es">Spanish</option>
                    <option value="fr">French</option>
                    <!-- Add more as needed -->
                </select>
            </div>
            <div>
                <label class="block text-sm mb-1">Year Range</label>
                <div class="flex gap-2">
                    <input type="number" id="filter-year-min" placeholder="From" class="w-1/2 px-3 py-2 border rounded" min="1800" max="2100">
                    <input type="number" id="filter-year-max" placeholder="To" class="w-1/2 px-3 py-2 border rounded" min="1800" max="2100">
                </div>
            </div>
            <div class="flex items-end">
                <label class="flex items-center">
                    <input type="checkbox" id="filter-has-cover" class="mr-2">
                    <span class="text-sm">Has Cover</span>
                </label>
            </div>
        </div>
        <div class="mt-4">
            <button onclick="applyFilters()" class="btn-primary px-4 py-2 rounded">Apply Filters</button>
            <button onclick="clearFilters()" class="btn-secondary px-4 py-2 rounded ml-2">Clear All</button>
        </div>
    </details>
</div>

<!-- Active filters display -->
<div id="active-filters" class="hidden mb-4">
    <div class="flex flex-wrap gap-2">
        <!-- Active filter chips added here dynamically -->
    </div>
</div>

JavaScript for Filter Management:

// Add to templates/dashboard.templ script section

function applyFilters() {
    const filters = {
        author: document.getElementById('filter-author').value,
        genre: document.getElementById('filter-genre').value,
        series: document.getElementById('filter-series').value,
        language: document.getElementById('filter-language').value,
        yearMin: parseInt(document.getElementById('filter-year-min').value) || 0,
        yearMax: parseInt(document.getElementById('filter-year-max').value) || 0,
        hasCover: document.getElementById('filter-has-cover').checked
    };

    const sort = document.getElementById('sort-select').value;

    // Update URL for shareability
    const params = new URLSearchParams();
    params.set('library_id', currentLibrary.id);
    params.set('sort', sort);
    Object.entries(filters).forEach(([key, value]) => {
        if (value) params.set(`filters[${key}]`, value);
    });

    window.history.pushState({}, '', `?${params.toString()}`);

    loadMediaItemsWithFilters(filters, sort);
    displayActiveFilters(filters);
}

function loadMediaItemsWithFilters(filters, sort) {
    const params = new URLSearchParams();
    params.set('library_id', currentLibrary.id);
    params.set('sort', sort);
    params.set('author_filter', filters.author);
    params.set('genre_filter', filters.genre);
    params.set('series_filter', filters.series);
    params.set('year_min', filters.yearMin);
    params.set('year_max', filters.yearMax);
    params.set('has_cover', filters.hasCover);

    fetch(`/api/media-items/filtered?${params.toString()}`, {
        headers: {
            'Authorization': 'Bearer ' + localStorage.getItem('token'),
            'Content-Type': 'application/json'
        }
    })
    .then(response => response.json())
    .then(data => {
        mediaItems = data.data || data;
        renderMediaItems();
    });
}

function displayActiveFilters(filters) {
    const container = document.getElementById('active-filters');
    const chips = container.querySelector('.flex');
    chips.innerHTML = '';

    let hasFilters = false;
    Object.entries(filters).forEach(([key, value]) => {
        if (value && value !== 0) {
            hasFilters = true;
            const chip = document.createElement('span');
            chip.className = 'px-3 py-1 rounded-full text-sm flex items-center gap-2';
            chip.style.cssText = 'background-color: var(--accent); color: var(--bg-primary)';
            chip.innerHTML = `${key}: ${value} <button onclick="clearFilter('${key}')" class="font-bold">×</button>`;
            chips.appendChild(chip);
        }
    });

    container.classList.toggle('hidden', !hasFilters);
}

function clearFilters() {
    document.getElementById('filter-author').value = '';
    document.getElementById('filter-genre').value = '';
    document.getElementById('filter-series').value = '';
    document.getElementById('filter-language').value = '';
    document.getElementById('filter-year-min').value = '';
    document.getElementById('filter-year-max').value = '';
    document.getElementById('filter-has-cover').checked = false;

    applyFilters();
}

// Load filters from URL on page load
document.addEventListener('DOMContentLoaded', function() {
    const params = new URLSearchParams(window.location.search);
    if (params.has('filters[author]')) {
        document.getElementById('filter-author').value = params.get('filters[author]');
    }
    // ... load other filters

    if (params.has('sort')) {
        document.getElementById('sort-select').value = params.get('sort');
    }
});

---

📁 KEY FILES & LOCATIONS

Database:

• database/schema/schema.sql - Main schema (merge changes here)
• internal/database/queries/queries.sql - SQL queries (add new ones)
• internal/database/queries.sql.go - Auto-generated by sqlc

Handlers:

• internal/handlers/ebook.go:824-865 - ListMediaItems endpoint
• internal/handlers/ebook.go:1673-1728 - SearchMediaItems (working reference)

Templates:

• templates/dashboard.templ:102-107 - Sort dropdown UI
• templates/dashboard.templ:222 - API call with sort param
• templates/dashboard.templ:371-383 - Client-side filter (needs replacement)
• templates/bookshelf.templ - Bookshelf view
• templates/header.templ - Search box

Tests:

• cmd/server/tests/search_test.go - Reference for test structure
• bruno/media-items/Search Media Items.bru - API documentation

Static Files:

• web/static/search.js - Search implementation (working reference)
• web/static/header.js - Header functionality

---

🔄 IMPLEMENTATION CHECKLIST

Phase 1: Add Database Fields

• Update database/schema/schema.sql with new columns
• Add indexes for new columns
• Delete current database
• Rebuild with clean Podman cache: podman-compose down -v && podman-compose up --build
• Run sqlc generate to update Go code
• Update scanner handlers to populate new fields
• Test with sample data

Phase 2: Fix Sorting

• Add ListMediaItemsSorted query to queries.sql
• Regenerate with sqlc generate
• Update ListMediaItems handler in ebook.go
• Update sort dropdown options in dashboard.templ
• Create Bruno test for sorting
• Write tests for no user, user, admin contexts
• Update README.md if needed

Phase 3: Add Filtering

• Add ListMediaItemsFiltered query to queries.sql
• Create handler endpoint for filtered results
• Add filter UI to dashboard.templ
• Implement JavaScript filter logic
• Add URL state management
• Create Bruno tests for filters
• Test all filter combinations
• Update README.md

---

🧪 TESTING REQUIREMENTS

Each phase must include tests covering:

1. No user context (unauthenticated requests)
2. User context (regular user with library visibility)
3. Admin context (admin with full access)

Example test structure (from search_test.go):

t.Run("User context - sorted results", func(t *testing.T) {
    req := httptest.NewRequest("GET", "/api/media-items?library_id=xxx&sort=title+ASC", nil)
    req.Header.Set("Authorization", "Bearer valid-user-token")
    // ... assertions
})

---

📊 CURRENT STATE REFERENCE

Working Search API:

GET /api/media-items/search?q=harry
Response: Array of media items with highlighted matches
Status: 200 (results), 404 (no results), 401 (unauthorized)

Broken Sort API:

GET /api/media-items?library_id=xxx&sort=title+ASC
Current behavior: Ignores sort parameter, returns default order
Expected behavior: Returns results sorted by title ASC

Existing Schema (23 fields):

id, library_id, title, author, isbn, description, file_path, file_size, mime_type, cover_image_path, series, series_number, tags, asin, date_published, publisher, contributors, added_by_admin_id, created_at, updated_at

Recommended New Fields (9 fields):

language, edition, page_count, goodreads_id, openlibrary_id, google_books_id, copyright_year, genre, subjects

---

⚠️ IMPORTANT NOTES

1. DO NOT create new migration files - merge into existing schema.sql
2. Delete and rebuild database when schema changes (per your standards)
3. Use Podman for all container operations
4. Include Bruno tests for all API changes
5. Follow pgx v5 patterns throughout
6. Multiple git commits preferred (one logical change per commit)
7. Test all three contexts (no user, user, admin)
8. Preserve existing functionality - search is working, don't break it

---

🚀 QUICK START COMMANDS

# After schema changes, rebuild database
podman-compose down -v
podman-compose up --build

# Regenerate Go database code
cd internal/database && sqlc generate

# Run tests
go test ./cmd/server/tests/...

# Access application
# URL: http://localhost:8765

---

📝 NEXT STEPS (When Resuming)

1. Start with Phase 1 (add database fields)
2. Follow PROJECT STANDARDS strictly
3. Create git commits after each working phase
4. Update this summary as you progress
5. Run tests frequently to catch regressions early

---

End of Summary