bookhoard/REMOVE_EBOOKS_SYSTEM.md

# Remove Ebooks System - Complete Migration to Media-Items

**Date:** January 30, 2026
**Status:** Ready to Execute
**Approach:** Complete removal (not deprecation)

## Database State Analysis

### Good News: Database is Already Clean!

The database schema **already uses media_items as the single source of truth**:

- ❌ **NO `ebooks` table exists** - it's a VIEW, not a table
- ✅ `media_items` table has ALL ebook fields PLUS more:
  - All ebook fields: title, author, isbn, description, file_path, file_size, mime_type, cover_image_path, series, series_number, tags, asin, date_published, publisher, contributors
  - **Additional fields not in old ebook system:** language, edition, page_count, goodreads_id, openlibrary_id, google_books_id, copyright_year, genre, subjects
- ✅ All user content already uses `media_item_id`:
  - reading_progress (media_item_id)
  - media_ratings (media_item_id)
  - media_notes (media_item_id)
  - media_highlights (media_item_id)

### What Exists (to be removed):

**Views to Delete:**
```sql
-- These are backward compatibility views - no longer needed
CREATE VIEW ebooks AS ...;
CREATE VIEW ebook_ratings AS ...;
CREATE VIEW ebook_reading_progress AS ...;
CREATE VIEW ebook_notes AS ...;
CREATE VIEW ebook_highlights AS ...;
```

**Database Queries to Remove:**
- All `GetEbook*`, `ListEbooks`, `CreateEbook`, `UpdateEbook`, `DeleteEbook` queries
- These query the VIEW, we should query media_items directly

## Code Changes Required

### Phase 1: Database Schema Cleanup

**File:** `database/schema/schema.sql`

**Actions:**
1. Remove 5 backward compatibility VIEW definitions (lines 124-217)
2. Keep all media_items tables and indexes
3. No migration needed - views are just queries, not data

**SQL to Remove:**
```sql
-- Lines 124-130: ebooks view
CREATE VIEW ebooks AS ...;

-- Lines 143-151: ebook_reading_progress view
CREATE VIEW ebook_reading_progress AS ...;

-- Lines 189-197: ebook_ratings view
CREATE VIEW ebook_ratings AS ...;

-- Lines 199-207: ebook_notes view
CREATE VIEW ebook_notes AS ...;

-- Lines 209-217: ebook_highlights view
CREATE VIEW ebook_highlights AS ...;
```

### Phase 2: Remove Database Queries

**File:** `internal/database/queries/queries.sql`

**Queries to Remove:**
- `GetEbook` - use `GetMediaItem` instead
- `ListEbooks` - use `ListMediaItems` or `ListMediaItemsFiltered` instead
- `CreateEbook` - replace with `CreateMediaItem`
- `UpdateEbook` - replace with `UpdateMediaItem`
- `DeleteEbook` - replace with `DeleteMediaItem`
- `GetEbookLibraryID` - no longer needed
- `GetEbookRating` - use `GetMediaRating` instead
- `CreateEbookRating` - use `CreateMediaRating` instead
- `DeleteEbookRating` - use `DeleteMediaRating` instead
- `GetEbookRatings` - use query on media_ratings instead
- `GetEbookNotes` - use `GetMediaNotes` instead
- `CreateEbookNote` - use `CreateMediaNote` instead
- `GetEbookNote` - use `GetMediaNote` instead
- `UpdateEbookNote` - use `UpdateMediaNote` instead
- `DeleteEbookNote` - use `DeleteMediaNote` instead
- `GetEbookHighlights` - use `GetMediaHighlights` instead
- `CreateEbookHighlight` - use `CreateMediaHighlight` instead
- `GetEbookHighlight` - use `GetMediaHighlight` instead
- `UpdateEbookHighlight` - use `UpdateMediaHighlight` instead
- `DeleteEbookHighlight` - use `DeleteMediaHighlight` instead

**Queries to Add:**
- `CreateMediaItem` - insert into media_items with library_id
- `UpdateMediaItem` - update media_items
- `DeleteMediaItem` - delete from media_items

### Phase 3: Remove API Endpoints

**File:** `internal/handlers/ebook.go`

**Routes to Remove from SetupRoutes():**
```go
// Remove ALL of these:
g.GET("/ebooks", h.ListEbooks)
g.GET("/ebooks/:id", h.GetEbook)
g.GET("/ebooks/:id/progress", h.GetReadingProgress)
g.PUT("/ebooks/:id/progress", h.UpdateReadingProgress)
g.GET("/ebooks/:id/rating", h.GetEbookRating)
g.POST("/ebooks/:id/rating", h.CreateOrUpdateEbookRating)
g.PUT("/ebooks/:id/rating", h.CreateOrUpdateEbookRating)
g.DELETE("/ebooks/:id/rating", h.DeleteEbookRating)
g.GET("/ebooks/:id/ratings", h.GetEbookRatings)
g.GET("/ebooks/:id/notes", h.GetEbookNotes)
g.POST("/ebooks/:id/notes", h.CreateEbookNote)
g.GET("/ebooks/:id/notes/:noteId", h.GetEbookNote)
g.PUT("/ebooks/:id/notes/:noteId", h.UpdateEbookNote)
g.DELETE("/ebooks/:id/notes/:noteId", h.DeleteEbookNote)
g.GET("/ebooks/:id/highlights", h.GetEbookHighlights)
g.POST("/ebooks/:id/highlights", h.CreateEbookHighlight)
g.GET("/ebooks/:id/highlights/:highlightId", h.GetEbookHighlight)
g.PUT("/ebooks/:id/highlights/:highlightId", h.UpdateEbookHighlight)
g.DELETE("/ebooks/:id/highlights/:highlightId", h.DeleteEbookHighlight)
admin.POST("/ebooks", h.CreateEbook)
admin.PUT("/ebooks/:id", h.UpdateEbook)
admin.DELETE("/ebooks", h.DeleteEbook)
```

**Routes to Add:**
```go
// Add these admin routes:
admin.POST("/media-items", h.CreateMediaItem)
admin.PUT("/media-items/:id", h.UpdateMediaItem)
admin.DELETE("/media-items/:id", h.DeleteMediaItem)
```

**Handlers to Remove:**
- `ListEbooks` - use ListMediaItems
- `GetEbook` - use GetMediaItem
- `CreateEbook` - replace with CreateMediaItem
- `UpdateEbook` - replace with UpdateMediaItem
- `DeleteEbook` - replace with DeleteMediaItem
- `GetEbookRating` - use GetMediaRating
- `CreateOrUpdateEbookRating` - use CreateMediaRating
- `DeleteEbookRating` - use DeleteMediaRating
- `GetEbookRatings` - use query on media_ratings
- `GetEbookNotes` - use GetMediaNotes
- `CreateEbookNote` - use CreateMediaNote
- `GetEbookNote` - use GetMediaNote
- `UpdateEbookNote` - use UpdateMediaNote
- `DeleteEbookNote` - use DeleteMediaNote
- `GetEbookHighlights` - use GetMediaHighlights
- `CreateEbookHighlight` - use CreateMediaHighlight
- `GetEbookHighlight` - use GetMediaHighlight
- `UpdateEbookHighlight` - use UpdateMediaHighlight
- `DeleteEbookHighlight` - use DeleteMediaHighlight

**Handlers to Add:**
- `CreateMediaItem(c echo.Context) error` - admin only, requires library_id
- `UpdateMediaItem(c echo.Context) error` - admin only
- `DeleteMediaItem(c echo.Context) error` - admin only

### Phase 4: Update Integration Tests

**File:** `cmd/server/tests/integration_test.go`

**Test Cases to Update:**
- Rename all "Ebooks" test cases to "MediaItems"
- Update API paths from `/api/ebooks` to `/api/media-items`
- Update response field names if needed
- Test the 3 new admin endpoints (Create, Update, Delete)

## Implementation Steps

### Step 1: Schema Cleanup (5 minutes)
```bash
# Edit database/schema/schema.sql
# Remove lines 124-217 (5 VIEW definitions)
# Keep all media_items tables
```

### Step 2: Update Database Queries (20 minutes)
```bash
# Regenerate database code
cd internal/database
sqlc generate
```

Need to create these new queries first in `queries.sql`:
```sql
-- Create media item
CREATE OR REPLACE FUNCTION CreateMediaItem(
    p_library_id UUID,
    p_title VARCHAR(255),
    p_author VARCHAR(255),
    p_isbn VARCHAR(17),
    p_description TEXT,
    p_file_path VARCHAR(500),
    p_file_size BIGINT,
    p_mime_type VARCHAR(100),
    p_cover_image_path VARCHAR(500),
    p_series VARCHAR(255),
    p_series_number INTEGER,
    p_tags TEXT,
    p_asin VARCHAR(20),
    p_date_published DATE,
    p_publisher VARCHAR(255),
    p_contributors TEXT,
    p_language VARCHAR(10),
    p_edition VARCHAR(255),
    p_page_count INTEGER,
    p_goodreads_id VARCHAR(20),
    p_openlibrary_id VARCHAR(100),
    p_google_books_id VARCHAR(100),
    p_copyright_year INTEGER,
    p_genre VARCHAR(100),
    p_subjects TEXT[],
    p_added_by_admin_id UUID
) RETURNS UUID AS $$
DECLARE
    v_media_item_id UUID;
BEGIN
    INSERT INTO media_items (
        library_id, title, author, isbn, description, file_path, file_size,
        mime_type, cover_image_path, series, series_number, tags, asin,
        date_published, publisher, contributors, language, edition, page_count,
        goodreads_id, openlibrary_id, google_books_id, copyright_year,
        genre, subjects, added_by_admin_id
    ) VALUES (
        p_library_id, p_title, p_author, p_isbn, p_description, p_file_path, p_file_size,
        p_mime_type, p_cover_image_path, p_series, p_series_number, p_tags, p_asin,
        p_date_published, p_publisher, p_contributors, p_language, p_edition, p_page_count,
        p_goodreads_id, p_openlibrary_id, p_google_books_id, p_copyright_year,
        p_genre, p_subjects, p_added_by_admin_id
    ) RETURNING id INTO v_media_item_id;

    RETURN v_media_item_id;
END;
$$ LANGUAGE plpgsql;

-- Update media item
CREATE OR REPLACE FUNCTION UpdateMediaItem(
    p_id UUID,
    p_title VARCHAR(255),
    p_author VARCHAR(255),
    p_isbn VARCHAR(17),
    p_description TEXT,
    p_cover_image_path VARCHAR(500),
    p_series VARCHAR(255),
    p_series_number INTEGER,
    p_tags TEXT,
    p_asin VARCHAR(20),
    p_date_published DATE,
    p_publisher VARCHAR(255),
    p_contributors TEXT,
    p_language VARCHAR(10),
    p_edition VARCHAR(255),
    p_page_count INTEGER,
    p_goodreads_id VARCHAR(20),
    p_openlibrary_id VARCHAR(100),
    p_google_books_id VARCHAR(100),
    p_copyright_year INTEGER,
    p_genre VARCHAR(100),
    p_subjects TEXT[]
) RETURNS BOOLEAN AS $$
BEGIN
    UPDATE media_items SET
        title = p_title,
        author = p_author,
        isbn = p_isbn,
        description = p_description,
        cover_image_path = p_cover_image_path,
        series = p_series,
        series_number = p_series_number,
        tags = p_tags,
        asin = p_asin,
        date_published = p_date_published,
        publisher = p_publisher,
        contributors = p_contributors,
        language = p_language,
        edition = p_edition,
        page_count = p_page_count,
        goodreads_id = p_goodreads_id,
        openlibrary_id = p_openlibrary_id,
        google_books_id = p_google_books_id,
        copyright_year = p_copyright_year,
        genre = p_genre,
        subjects = p_subjects,
        updated_at = NOW()
    WHERE id = p_id;

    RETURN FOUND;
END;
$$ LANGUAGE plpgsql;

-- Delete media item
CREATE OR REPLACE FUNCTION DeleteMediaItem(p_id UUID) RETURNS BOOLEAN AS $$
BEGIN
    DELETE FROM media_items WHERE id = p_id;
    RETURN FOUND;
END;
$$ LANGUAGE plpgsql;
```

### Step 3: Remove Old Handlers (30 minutes)
```bash
# Delete these handler functions from internal/handlers/ebook.go:
# - ListEbooks
# - GetEbook
# - CreateEbook
# - UpdateEbook
# - DeleteEbook
# - GetEbookRating, CreateOrUpdateEbookRating, DeleteEbookRating, GetEbookRatings
# - GetEbookNotes, CreateEbookNote, GetEbookNote, UpdateEbookNote, DeleteEbookNote
# - GetEbookHighlights, CreateEbookHighlight, GetEbookHighlight, UpdateEbookHighlight, DeleteEbookHighlight
```

### Step 4: Add New Admin Handlers (20 minutes)
```go
// CreateMediaItem - admin only, requires library_id
func (h *Handler) CreateMediaItem(c echo.Context) error {
    user := MustGetAuthenticatedUser(c)

    // Verify user is admin
    if user.Role != "admin" {
        return c.JSON(http.StatusForbidden, map[string]string{"error": "admin access required"})
    }

    var req CreateMediaItemRequest
    if err := c.Bind(&req); err != nil {
        return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid request"})
    }
    if err := c.Validate(&req); err != nil {
        return c.JSON(http.StatusBadRequest, map[string]string{"error": err.Error()})
    }

    // Validate library exists
    _, err := h.db.GetLibrary(c.Request().Context(), req.LibraryID)
    if err != nil {
        if err == pgx.ErrNoRows {
            return c.JSON(http.StatusBadRequest, map[string]string{"error": "library not found"})
        }
        return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
    }

    mediaItemID, err := h.db.CreateMediaItem(c.Request().Context(), database.CreateMediaItemParams{
        LibraryID:       req.LibraryID,
        Title:           req.Title,
        Author:          pgtype.Text{String: req.Author, Valid: req.Author != ""},
        Isbn:            req.ISBN,
        Description:     pgtype.Text{String: req.Description, Valid: req.Description != ""},
        FilePath:        req.FilePath,
        FileSize:        pgtype.Int8{Int64: req.FileSize, Valid: req.FileSize > 0},
        MimeType:        pgtype.Text{String: req.MimeType, Valid: req.MimeType != ""},
        CoverImagePath:  pgtype.Text{String: req.CoverImagePath, Valid: req.CoverImagePath != ""},
        Series:          pgtype.Text{String: req.Series, Valid: req.Series != ""},
        SeriesNumber:    pgtype.Int4{Int32: req.SeriesNumber, Valid: req.SeriesNumber > 0},
        Tags:            pgtype.Text{String: req.Tags, Valid: req.Tags != ""},
        Asin:            pgtype.Text{String: req.ASIN, Valid: req.ASIN != ""},
        DatePublished:   pgtype.Date{Time: parseDate(req.DatePublished), Valid: req.DatePublished != ""},
        Publisher:       pgtype.Text{String: req.Publisher, Valid: req.Publisher != ""},
        Contributors:    pgtype.Text{String: req.Contributors, Valid: req.Contributors != ""},
        AddedByAdminID:  user.ID,
    })
    if err != nil {
        return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
    }

    // Fetch the created item
    item, err := h.db.GetMediaItem(c.Request().Context(), mediaItemID)
    if err != nil {
        return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
    }

    return c.JSON(http.StatusCreated, item)
}

type CreateMediaItemRequest struct {
    LibraryID      uuid.UUID `json:"library_id" validate:"required"`
    Title          string    `json:"title" validate:"required,min=1,max=500"`
    Author         string    `json:"author"`
    ISBN           string    `json:"isbn"`
    Description    string    `json:"description"`
    FilePath       string    `json:"file_path" validate:"required"`
    FileSize       int64     `json:"file_size" validate:"required,min=1"`
    MimeType       string    `json:"mime_type" validate:"required"`
    CoverImagePath string    `json:"cover_image_path"`
    Series         string    `json:"series"`
    SeriesNumber   int32     `json:"series_number"`
    Tags           string    `json:"tags"`
    ASIN           string    `json:"asin"`
    DatePublished  string    `json:"date_published"`
    Publisher      string    `json:"publisher"`
    Contributors   string    `json:"contributors"`
}

// UpdateMediaItem and DeleteMediaItem similar pattern
```

### Step 5: Update SetupRoutes (5 minutes)
```go
func SetupRoutes(g *echo.Group, db *database.Queries) *Handler {
    h := NewHandler(db)

    // Media item routes (all authenticated users)
    g.GET("/media-items", h.ListMediaItems)
    g.GET("/media-items/filtered", h.ListMediaItemsFiltered)
    g.GET("/media-items/search", h.SearchMediaItems)
    g.GET("/media-items/:id", h.GetMediaItem)

    // Admin-only routes
    admin := g.Group("", AdminMiddleware)
    admin.POST("/media-items", h.CreateMediaItem)
    admin.PUT("/media-items/:id", h.UpdateMediaItem)
    admin.DELETE("/media-items/:id", h.DeleteMediaItem)

    // ... rest of routes (ratings, progress, notes, highlights, scanner)

    return h
}
```

### Step 6: Update Tests (20 minutes)
```bash
# Rename test cases from "Ebooks" to "MediaItems"
# Update API paths
# Test new admin endpoints
```

### Step 7: Rebuild and Test (10 minutes)
```bash
podman compose down -v
podman volume prune -f
podman compose build --no-cache
podman compose up -d
go test -v ./cmd/server/tests -run TestIntegrationAPI
```

## Files to Modify

1. `database/schema/schema.sql` - Remove 5 VIEW definitions
2. `internal/database/queries/queries.sql` - Remove ebook queries, add media items queries
3. `internal/handlers/ebook.go` - Remove ebook handlers, add media items admin handlers
4. `cmd/server/tests/integration_test.go` - Update test cases
5. `API_TESTING_SUMMARY.md` - Update documentation

## Database Migration

**No migration needed!** Views are just queries, not tables. Removing them has no impact on data.

## Feature Parity Verification

### ✅ All Features Preserved in Media-Items

**Ebook Feature → Media-Items Equivalent:**
- `GET /api/ebooks` → `GET /api/media-items`
- `GET /api/ebooks/:id` → `GET /api/media-items/:id`
- `POST /api/ebooks` → `POST /api/media-items` (NEW)
- `PUT /api/ebooks/:id` → `PUT /api/media-items/:id` (NEW)
- `DELETE /api/ebooks/:id` → `DELETE /api/media-items/:id` (NEW)
- Rating endpoints → Already exist for media-items
- Progress endpoints → Already exist for media-items
- Notes endpoints → Already exist for media-items
- Highlights endpoints → Already exist for media-items

**Enhanced Features in Media-Items:**
- ✅ Library scoping (required library_id)
- ✅ Better filtering (by author, series, genre, language, year, cover)
- ✅ Advanced search (partial + fuzzy)
- ✅ More sorting options
- ✅ Additional metadata fields

## Time Estimate

- Schema cleanup: 5 minutes
- Database queries: 20 minutes
- Remove old handlers: 30 minutes
- Add new admin handlers: 20 minutes
- Update routes: 5 minutes
- Update tests: 20 minutes
- Rebuild & test: 10 minutes
- **Total: ~110 minutes (2 hours)**

## Success Criteria

✅ All /api/ebooks endpoints removed
✅ All backward compatibility views removed from schema
✅ Admin can create/update/delete media items
✅ All filtering, sorting, searching works
✅ All integration tests pass
✅ Clean database schema (no unused tables/views)
✅ Code is simpler (one system, not two)

## Risk Assessment

**Medium Risk:**
- Removing endpoints is a breaking change
- But app has never been released, so no external users
- Complete control over all API consumers

**Mitigation:**
- Bruno collections will need updating
- Integration tests provide safety net
- Can revert commit if issues found

---

**Status:** Ready to execute
**Interruptible:** Yes, but best to complete all phases
**Reversible:** Yes via git revert
**Database Impact:** No data loss (views only)