bookhoard/REMOVE_EBOOKS_SYSTEM.md

Remove Ebooks System - Complete Migration to Media-Items

Date: January 30, 2026 Status: Ready to Execute Approach: Complete removal (not deprecation)

Database State Analysis

Good News: Database is Already Clean!

The database schema already uses media_items as the single source of truth:

• ❌ NO ebooks table exists - it's a VIEW, not a table
• ✅ media_items table has ALL ebook fields PLUS more:
  • All ebook fields: title, author, isbn, description, file_path, file_size, mime_type, cover_image_path, series, series_number, tags, asin, date_published, publisher, contributors
  • Additional fields not in old ebook system: language, edition, page_count, goodreads_id, openlibrary_id, google_books_id, copyright_year, genre, subjects
• ✅ All user content already uses media_item_id:
  • reading_progress (media_item_id)
  • media_ratings (media_item_id)
  • media_notes (media_item_id)
  • media_highlights (media_item_id)

What Exists (to be removed):

Views to Delete:

-- These are backward compatibility views - no longer needed
CREATE VIEW ebooks AS ...;
CREATE VIEW ebook_ratings AS ...;
CREATE VIEW ebook_reading_progress AS ...;
CREATE VIEW ebook_notes AS ...;
CREATE VIEW ebook_highlights AS ...;

Database Queries to Remove:

• All GetEbook*, ListEbooks, CreateEbook, UpdateEbook, DeleteEbook queries
• These query the VIEW, we should query media_items directly

Code Changes Required

Phase 1: Database Schema Cleanup

File: database/schema/schema.sql

Actions:

1. Remove 5 backward compatibility VIEW definitions (lines 124-217)
2. Keep all media_items tables and indexes
3. No migration needed - views are just queries, not data

SQL to Remove:

-- Lines 124-130: ebooks view
CREATE VIEW ebooks AS ...;

-- Lines 143-151: ebook_reading_progress view
CREATE VIEW ebook_reading_progress AS ...;

-- Lines 189-197: ebook_ratings view
CREATE VIEW ebook_ratings AS ...;

-- Lines 199-207: ebook_notes view
CREATE VIEW ebook_notes AS ...;

-- Lines 209-217: ebook_highlights view
CREATE VIEW ebook_highlights AS ...;

Phase 2: Remove Database Queries

File: internal/database/queries/queries.sql

Queries to Remove:

• GetEbook - use GetMediaItem instead
• ListEbooks - use ListMediaItems or ListMediaItemsFiltered instead
• CreateEbook - replace with CreateMediaItem
• UpdateEbook - replace with UpdateMediaItem
• DeleteEbook - replace with DeleteMediaItem
• GetEbookLibraryID - no longer needed
• GetEbookRating - use GetMediaRating instead
• CreateEbookRating - use CreateMediaRating instead
• DeleteEbookRating - use DeleteMediaRating instead
• GetEbookRatings - use query on media_ratings instead
• GetEbookNotes - use GetMediaNotes instead
• CreateEbookNote - use CreateMediaNote instead
• GetEbookNote - use GetMediaNote instead
• UpdateEbookNote - use UpdateMediaNote instead
• DeleteEbookNote - use DeleteMediaNote instead
• GetEbookHighlights - use GetMediaHighlights instead
• CreateEbookHighlight - use CreateMediaHighlight instead
• GetEbookHighlight - use GetMediaHighlight instead
• UpdateEbookHighlight - use UpdateMediaHighlight instead
• DeleteEbookHighlight - use DeleteMediaHighlight instead

Queries to Add:

• CreateMediaItem - insert into media_items with library_id
• UpdateMediaItem - update media_items
• DeleteMediaItem - delete from media_items

Phase 3: Remove API Endpoints

File: internal/handlers/ebook.go

Routes to Remove from SetupRoutes():

// Remove ALL of these:
g.GET("/ebooks", h.ListEbooks)
g.GET("/ebooks/:id", h.GetEbook)
g.GET("/ebooks/:id/progress", h.GetReadingProgress)
g.PUT("/ebooks/:id/progress", h.UpdateReadingProgress)
g.GET("/ebooks/:id/rating", h.GetEbookRating)
g.POST("/ebooks/:id/rating", h.CreateOrUpdateEbookRating)
g.PUT("/ebooks/:id/rating", h.CreateOrUpdateEbookRating)
g.DELETE("/ebooks/:id/rating", h.DeleteEbookRating)
g.GET("/ebooks/:id/ratings", h.GetEbookRatings)
g.GET("/ebooks/:id/notes", h.GetEbookNotes)
g.POST("/ebooks/:id/notes", h.CreateEbookNote)
g.GET("/ebooks/:id/notes/:noteId", h.GetEbookNote)
g.PUT("/ebooks/:id/notes/:noteId", h.UpdateEbookNote)
g.DELETE("/ebooks/:id/notes/:noteId", h.DeleteEbookNote)
g.GET("/ebooks/:id/highlights", h.GetEbookHighlights)
g.POST("/ebooks/:id/highlights", h.CreateEbookHighlight)
g.GET("/ebooks/:id/highlights/:highlightId", h.GetEbookHighlight)
g.PUT("/ebooks/:id/highlights/:highlightId", h.UpdateEbookHighlight)
g.DELETE("/ebooks/:id/highlights/:highlightId", h.DeleteEbookHighlight)
admin.POST("/ebooks", h.CreateEbook)
admin.PUT("/ebooks/:id", h.UpdateEbook)
admin.DELETE("/ebooks", h.DeleteEbook)

Routes to Add:

// Add these admin routes:
admin.POST("/media-items", h.CreateMediaItem)
admin.PUT("/media-items/:id", h.UpdateMediaItem)
admin.DELETE("/media-items/:id", h.DeleteMediaItem)

Handlers to Remove:

• ListEbooks - use ListMediaItems
• GetEbook - use GetMediaItem
• CreateEbook - replace with CreateMediaItem
• UpdateEbook - replace with UpdateMediaItem
• DeleteEbook - replace with DeleteMediaItem
• GetEbookRating - use GetMediaRating
• CreateOrUpdateEbookRating - use CreateMediaRating
• DeleteEbookRating - use DeleteMediaRating
• GetEbookRatings - use query on media_ratings
• GetEbookNotes - use GetMediaNotes
• CreateEbookNote - use CreateMediaNote
• GetEbookNote - use GetMediaNote
• UpdateEbookNote - use UpdateMediaNote
• DeleteEbookNote - use DeleteMediaNote
• GetEbookHighlights - use GetMediaHighlights
• CreateEbookHighlight - use CreateMediaHighlight
• GetEbookHighlight - use GetMediaHighlight
• UpdateEbookHighlight - use UpdateMediaHighlight
• DeleteEbookHighlight - use DeleteMediaHighlight

Handlers to Add:

• CreateMediaItem(c echo.Context) error - admin only, requires library_id
• UpdateMediaItem(c echo.Context) error - admin only
• DeleteMediaItem(c echo.Context) error - admin only

Phase 4: Update Integration Tests

File: cmd/server/tests/integration_test.go

Test Cases to Update:

• Rename all "Ebooks" test cases to "MediaItems"
• Update API paths from /api/ebooks to /api/media-items
• Update response field names if needed
• Test the 3 new admin endpoints (Create, Update, Delete)

Implementation Steps

Step 1: Schema Cleanup (5 minutes)

# Edit database/schema/schema.sql
# Remove lines 124-217 (5 VIEW definitions)
# Keep all media_items tables

Step 2: Update Database Queries (20 minutes)

# Regenerate database code
cd internal/database
sqlc generate

Need to create these new queries first in queries.sql:

-- Create media item
CREATE OR REPLACE FUNCTION CreateMediaItem(
    p_library_id UUID,
    p_title VARCHAR(255),
    p_author VARCHAR(255),
    p_isbn VARCHAR(17),
    p_description TEXT,
    p_file_path VARCHAR(500),
    p_file_size BIGINT,
    p_mime_type VARCHAR(100),
    p_cover_image_path VARCHAR(500),
    p_series VARCHAR(255),
    p_series_number INTEGER,
    p_tags TEXT,
    p_asin VARCHAR(20),
    p_date_published DATE,
    p_publisher VARCHAR(255),
    p_contributors TEXT,
    p_language VARCHAR(10),
    p_edition VARCHAR(255),
    p_page_count INTEGER,
    p_goodreads_id VARCHAR(20),
    p_openlibrary_id VARCHAR(100),
    p_google_books_id VARCHAR(100),
    p_copyright_year INTEGER,
    p_genre VARCHAR(100),
    p_subjects TEXT[],
    p_added_by_admin_id UUID
) RETURNS UUID AS $$
DECLARE
    v_media_item_id UUID;
BEGIN
    INSERT INTO media_items (
        library_id, title, author, isbn, description, file_path, file_size,
        mime_type, cover_image_path, series, series_number, tags, asin,
        date_published, publisher, contributors, language, edition, page_count,
        goodreads_id, openlibrary_id, google_books_id, copyright_year,
        genre, subjects, added_by_admin_id
    ) VALUES (
        p_library_id, p_title, p_author, p_isbn, p_description, p_file_path, p_file_size,
        p_mime_type, p_cover_image_path, p_series, p_series_number, p_tags, p_asin,
        p_date_published, p_publisher, p_contributors, p_language, p_edition, p_page_count,
        p_goodreads_id, p_openlibrary_id, p_google_books_id, p_copyright_year,
        p_genre, p_subjects, p_added_by_admin_id
    ) RETURNING id INTO v_media_item_id;

    RETURN v_media_item_id;
END;
$$ LANGUAGE plpgsql;

-- Update media item
CREATE OR REPLACE FUNCTION UpdateMediaItem(
    p_id UUID,
    p_title VARCHAR(255),
    p_author VARCHAR(255),
    p_isbn VARCHAR(17),
    p_description TEXT,
    p_cover_image_path VARCHAR(500),
    p_series VARCHAR(255),
    p_series_number INTEGER,
    p_tags TEXT,
    p_asin VARCHAR(20),
    p_date_published DATE,
    p_publisher VARCHAR(255),
    p_contributors TEXT,
    p_language VARCHAR(10),
    p_edition VARCHAR(255),
    p_page_count INTEGER,
    p_goodreads_id VARCHAR(20),
    p_openlibrary_id VARCHAR(100),
    p_google_books_id VARCHAR(100),
    p_copyright_year INTEGER,
    p_genre VARCHAR(100),
    p_subjects TEXT[]
) RETURNS BOOLEAN AS $$
BEGIN
    UPDATE media_items SET
        title = p_title,
        author = p_author,
        isbn = p_isbn,
        description = p_description,
        cover_image_path = p_cover_image_path,
        series = p_series,
        series_number = p_series_number,
        tags = p_tags,
        asin = p_asin,
        date_published = p_date_published,
        publisher = p_publisher,
        contributors = p_contributors,
        language = p_language,
        edition = p_edition,
        page_count = p_page_count,
        goodreads_id = p_goodreads_id,
        openlibrary_id = p_openlibrary_id,
        google_books_id = p_google_books_id,
        copyright_year = p_copyright_year,
        genre = p_genre,
        subjects = p_subjects,
        updated_at = NOW()
    WHERE id = p_id;

    RETURN FOUND;
END;
$$ LANGUAGE plpgsql;

-- Delete media item
CREATE OR REPLACE FUNCTION DeleteMediaItem(p_id UUID) RETURNS BOOLEAN AS $$
BEGIN
    DELETE FROM media_items WHERE id = p_id;
    RETURN FOUND;
END;
$$ LANGUAGE plpgsql;

Step 3: Remove Old Handlers (30 minutes)

# Delete these handler functions from internal/handlers/ebook.go:
# - ListEbooks
# - GetEbook
# - CreateEbook
# - UpdateEbook
# - DeleteEbook
# - GetEbookRating, CreateOrUpdateEbookRating, DeleteEbookRating, GetEbookRatings
# - GetEbookNotes, CreateEbookNote, GetEbookNote, UpdateEbookNote, DeleteEbookNote
# - GetEbookHighlights, CreateEbookHighlight, GetEbookHighlight, UpdateEbookHighlight, DeleteEbookHighlight

Step 4: Add New Admin Handlers (20 minutes)

// CreateMediaItem - admin only, requires library_id
func (h *Handler) CreateMediaItem(c echo.Context) error {
    user := MustGetAuthenticatedUser(c)

    // Verify user is admin
    if user.Role != "admin" {
        return c.JSON(http.StatusForbidden, map[string]string{"error": "admin access required"})
    }

    var req CreateMediaItemRequest
    if err := c.Bind(&req); err != nil {
        return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid request"})
    }
    if err := c.Validate(&req); err != nil {
        return c.JSON(http.StatusBadRequest, map[string]string{"error": err.Error()})
    }

    // Validate library exists
    _, err := h.db.GetLibrary(c.Request().Context(), req.LibraryID)
    if err != nil {
        if err == pgx.ErrNoRows {
            return c.JSON(http.StatusBadRequest, map[string]string{"error": "library not found"})
        }
        return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
    }

    mediaItemID, err := h.db.CreateMediaItem(c.Request().Context(), database.CreateMediaItemParams{
        LibraryID:       req.LibraryID,
        Title:           req.Title,
        Author:          pgtype.Text{String: req.Author, Valid: req.Author != ""},
        Isbn:            req.ISBN,
        Description:     pgtype.Text{String: req.Description, Valid: req.Description != ""},
        FilePath:        req.FilePath,
        FileSize:        pgtype.Int8{Int64: req.FileSize, Valid: req.FileSize > 0},
        MimeType:        pgtype.Text{String: req.MimeType, Valid: req.MimeType != ""},
        CoverImagePath:  pgtype.Text{String: req.CoverImagePath, Valid: req.CoverImagePath != ""},
        Series:          pgtype.Text{String: req.Series, Valid: req.Series != ""},
        SeriesNumber:    pgtype.Int4{Int32: req.SeriesNumber, Valid: req.SeriesNumber > 0},
        Tags:            pgtype.Text{String: req.Tags, Valid: req.Tags != ""},
        Asin:            pgtype.Text{String: req.ASIN, Valid: req.ASIN != ""},
        DatePublished:   pgtype.Date{Time: parseDate(req.DatePublished), Valid: req.DatePublished != ""},
        Publisher:       pgtype.Text{String: req.Publisher, Valid: req.Publisher != ""},
        Contributors:    pgtype.Text{String: req.Contributors, Valid: req.Contributors != ""},
        AddedByAdminID:  user.ID,
    })
    if err != nil {
        return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
    }

    // Fetch the created item
    item, err := h.db.GetMediaItem(c.Request().Context(), mediaItemID)
    if err != nil {
        return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
    }

    return c.JSON(http.StatusCreated, item)
}

type CreateMediaItemRequest struct {
    LibraryID      uuid.UUID `json:"library_id" validate:"required"`
    Title          string    `json:"title" validate:"required,min=1,max=500"`
    Author         string    `json:"author"`
    ISBN           string    `json:"isbn"`
    Description    string    `json:"description"`
    FilePath       string    `json:"file_path" validate:"required"`
    FileSize       int64     `json:"file_size" validate:"required,min=1"`
    MimeType       string    `json:"mime_type" validate:"required"`
    CoverImagePath string    `json:"cover_image_path"`
    Series         string    `json:"series"`
    SeriesNumber   int32     `json:"series_number"`
    Tags           string    `json:"tags"`
    ASIN           string    `json:"asin"`
    DatePublished  string    `json:"date_published"`
    Publisher      string    `json:"publisher"`
    Contributors   string    `json:"contributors"`
}

// UpdateMediaItem and DeleteMediaItem similar pattern

Step 5: Update SetupRoutes (5 minutes)

func SetupRoutes(g *echo.Group, db *database.Queries) *Handler {
    h := NewHandler(db)

    // Media item routes (all authenticated users)
    g.GET("/media-items", h.ListMediaItems)
    g.GET("/media-items/filtered", h.ListMediaItemsFiltered)
    g.GET("/media-items/search", h.SearchMediaItems)
    g.GET("/media-items/:id", h.GetMediaItem)

    // Admin-only routes
    admin := g.Group("", AdminMiddleware)
    admin.POST("/media-items", h.CreateMediaItem)
    admin.PUT("/media-items/:id", h.UpdateMediaItem)
    admin.DELETE("/media-items/:id", h.DeleteMediaItem)

    // ... rest of routes (ratings, progress, notes, highlights, scanner)

    return h
}

Step 6: Update Tests (20 minutes)

# Rename test cases from "Ebooks" to "MediaItems"
# Update API paths
# Test new admin endpoints

Step 7: Rebuild and Test (10 minutes)

podman compose down -v
podman volume prune -f
podman compose build --no-cache
podman compose up -d
go test -v ./cmd/server/tests -run TestIntegrationAPI

Files to Modify

1. database/schema/schema.sql - Remove 5 VIEW definitions
2. internal/database/queries/queries.sql - Remove ebook queries, add media items queries
3. internal/handlers/ebook.go - Remove ebook handlers, add media items admin handlers
4. cmd/server/tests/integration_test.go - Update test cases
5. API_TESTING_SUMMARY.md - Update documentation

Database Migration

No migration needed! Views are just queries, not tables. Removing them has no impact on data.

Feature Parity Verification

✅ All Features Preserved in Media-Items

Ebook Feature → Media-Items Equivalent:

• GET /api/ebooks → GET /api/media-items
• GET /api/ebooks/:id → GET /api/media-items/:id
• POST /api/ebooks → POST /api/media-items (NEW)
• PUT /api/ebooks/:id → PUT /api/media-items/:id (NEW)
• DELETE /api/ebooks/:id → DELETE /api/media-items/:id (NEW)
• Rating endpoints → Already exist for media-items
• Progress endpoints → Already exist for media-items
• Notes endpoints → Already exist for media-items
• Highlights endpoints → Already exist for media-items

Enhanced Features in Media-Items:

• ✅ Library scoping (required library_id)
• ✅ Better filtering (by author, series, genre, language, year, cover)
• ✅ Advanced search (partial + fuzzy)
• ✅ More sorting options
• ✅ Additional metadata fields

Time Estimate

• Schema cleanup: 5 minutes
• Database queries: 20 minutes
• Remove old handlers: 30 minutes
• Add new admin handlers: 20 minutes
• Update routes: 5 minutes
• Update tests: 20 minutes
• Rebuild & test: 10 minutes
• Total: ~110 minutes (2 hours)

Success Criteria

✅ All /api/ebooks endpoints removed ✅ All backward compatibility views removed from schema ✅ Admin can create/update/delete media items ✅ All filtering, sorting, searching works ✅ All integration tests pass ✅ Clean database schema (no unused tables/views) ✅ Code is simpler (one system, not two)

Risk Assessment

Medium Risk:

• Removing endpoints is a breaking change
• But app has never been released, so no external users
• Complete control over all API consumers

Mitigation:

• Bruno collections will need updating
• Integration tests provide safety net
• Can revert commit if issues found

---

Status: Ready to execute Interruptible: Yes, but best to complete all phases Reversible: Yes via git revert Database Impact: No data loss (views only)