# Cover & File Serving - Implementation Plan ## Overview Fix file and cover image serving to support: 1. Multiple library folders in docker compose (flexible mount points) 2. Keep files with books (no hardcoded paths) 3. Store relative paths in database (for both files AND covers) 4. Serve everything via authenticated API endpoints 5. Mobile app compatibility (same auth for all requests) ## Architecture ### Current Behavior - File path stored as absolute: `/app/uploads/Jane Austen/Pride and Prejudice/book.epub` - Cover path stored as absolute: `/app/uploads/Jane Austen/Pride and Prejudice/cover.jpg` - Frontend uses path directly - doesn't work (browser can't access container paths) - No route serves `/app/uploads/*` ### Target Behavior - File path stored as relative: `Jane Austen/Pride and Prejudice/book.epub` - Cover path stored as relative: `Jane Austen/Pride and Prejudice/cover.jpg` - Handler resolves relative path using library folder base path - All file/cover requests go through authenticated API endpoints - Mobile app can use same endpoints with JWT auth ### URL Format To handle same relative paths in different libraries: ``` /api/files/{media_item_id}/content → serves the book file /api/covers/{media_item_id} → serves the cover image ``` --- ## Phase 1: Update Scanner to Store Relative Paths (Files AND Covers) ### File: `internal/services/media_scanner.go` #### Change 1: Store relative file path **Location**: Where metadata.FilePath is set (multiple locations) **Current code**: ```go metadata.FilePath = path // path is absolute like /app/uploads/Author/Book/file.epub ``` **New code**: ```go metadata.FilePath = s.getRelativePath(path) ``` #### Change 2: Store relative cover path **Location**: Around lines 514-517, 641-651, 832-837, 1056-1067, 1472 **Current code** (example at line 514-517): ```go if len(coverImage) > 0 && metadata.CoverPath == "" { coverPath := path + ".cover.jpg" if err := os.WriteFile(coverPath, coverImage, 0644); err == nil { metadata.CoverPath = coverPath } } ``` **New code**: ```go if len(coverImage) > 0 && metadata.CoverPath == "" { coverPath := path + ".cover.jpg" if err := os.WriteFile(coverPath, coverImage, 0644); err == nil { // Store relative path - derive from library folder base metadata.CoverPath = s.getRelativePath(coverPath) } } ``` #### Change 3: Add helper function **Add new function** in `internal/services/media_scanner.go`: ```go // getRelativePath converts absolute filesystem path to relative path // using the library folder base path func (s *MediaScanner) getRelativePath(absolutePath string) string { // Get the base folder paths from scanner for _, baseFolder := range s.folders { // Check if path is within this base folder if strings.HasPrefix(absolutePath, baseFolder) { // Return relative path (without leading slash) relPath := strings.TrimPrefix(absolutePath, baseFolder) // Remove leading slash if present relPath = strings.TrimPrefix(relPath, "/") return relPath } } // Fallback: if no match, return as-is (shouldn't happen) return absolutePath } ``` **Note**: This uses `s.folders` which is already populated in the scanner. #### Change 4: Update force rescan path handling **Location**: Around line 1472 (in the force rescan/update flow) Apply same `getRelativePath()` conversion when updating existing items. --- ## Phase 2: Create Path Resolution Helper (Service Layer) ### File: `internal/services/library_service.go` (or new file) Create a reusable function that resolves relative paths to absolute filesystem paths: ```go // ResolveMediaPath resolves a relative path to absolute filesystem path // using the library's configured folder(s) func (s *LibraryService) ResolveMediaPath(ctx context.Context, libraryID pgtype.UUID, relativePath string) (string, error) { // Get library folders for this library folders, err := s.db.GetLibraryFolders(ctx, libraryID) if err != nil || len(folders) == 0 { return "", fmt.Errorf("no library folders found for library") } // Try each folder - find one where the relative path makes sense for _, folder := range folders { fullPath := filepath.Join(folder.FolderPath, relativePath) if _, err := os.Stat(fullPath); err == nil { return fullPath, nil } } // Fallback: use first folder (file might not exist yet during scan) if len(folders) > 0 { return filepath.Join(folders[0].FolderPath, relativePath), nil } return "", fmt.Errorf("could not resolve path") } ``` --- ## Phase 3: Modify Existing Endpoints to Return Resolved URLs ### Strategy Instead of creating new endpoints, modify existing API responses to include resolved/usable URLs. This avoids extra HTTP calls. ### File: `internal/handlers/media.go` #### Update ListMediaItems response **Location**: Around line 612-620 Create a wrapper struct or modify the response to include resolved cover URLs: ```go // MediaItemResponse includes resolved URLs for frontend use type MediaItemResponse struct { database.ListMediaItemsRow CoverURL string `json:"cover_url"` FileURL string `json:"file_url"` } // In ListMediaItems handler: items, err := mh.db.ListMediaItems(...) // ... existing code ... // Transform to response with resolved URLs response := make([]MediaItemResponse, len(items)) for i, item := range items { response[i] = MediaItemResponse{ ListMediaItemsRow: item, CoverURL: mh.resolveCoverURL(item.LibraryID, item.CoverImagePath.String), } } ``` #### Add resolve helper methods ```go // resolveCoverURL returns a usable URL for the cover image func (mh *MediaHandler) resolveCoverURL(libraryID pgtype.UUID, coverPath string) string { if coverPath == "" { return "" } // For now, return empty - frontend uses existing cover endpoint // Later this could return /api/covers/{id} pattern return "" } // getFullFilePath returns the absolute filesystem path for a media item func (mh *MediaHandler) getFullFilePath(ctx context.Context, libraryID pgtype.UUID, relativePath string) (string, error) { if relativePath == "" { return "", fmt.Errorf("no file path") } // Check if already absolute (backward compatibility) if filepath.IsAbs(relativePath) { return relativePath, nil } // Resolve relative path using library folders return mh.libraryService.ResolveMediaPath(ctx, libraryID, relativePath) } ``` --- ## Phase 4: Update Download Handler to Use Relative Paths ### File: `internal/handlers/media.go` #### Modify DownloadBook function **Current code** (line 103-144): ```go func (h *MediaHandler) DownloadBook(c echo.Context) error { // ... mediaItem, err := h.db.GetMediaItem(c.Request().Context(), pgBookUUID) if err != nil { return c.JSON(http.StatusNotFound, map[string]string{"error": "book not found"}) } if _, err := os.Stat(mediaItem.FilePath); os.IsNotExist(err) { return c.JSON(http.StatusNotFound, map[string]string{"error": "book file not found on disk"}) } file, err := os.Open(mediaItem.FilePath) // ... } ``` **New code**: ```go func (h *MediaHandler) DownloadBook(c echo.Context) error { // ... mediaItem, err := h.db.GetMediaItem(c.Request().Context(), pgBookUUID) if err != nil { return c.JSON(http.StatusNotFound, map[string]string{"error": "book not found"}) } // Resolve relative path to absolute filesystem path fullPath, err := h.getFullFilePath(c.Request().Context(), mediaItem.LibraryID, mediaItem.FilePath) if err != nil { return c.JSON(http.StatusNotFound, map[string]string{"error": "book file not found on disk"}) } if _, err := os.Stat(fullPath); os.IsNotExist(err) { return c.JSON(http.StatusNotFound, map[string]string{"error": "book file not found on disk"}) } file, err := os.Open(fullPath) // ... } ``` --- ## Phase 5: Create Cover Image Endpoint (For Cases Needing Direct Access) ### File: `internal/handlers/media.go` While Phase 3 handles cover URLs in existing responses, this endpoint is useful for cases where you need direct cover access (e.g., dynamic updates, specific UI components). ```go // GetCoverImage handles GET /api/covers/:id // Returns the cover image for a media item func (mh *MediaHandler) GetCoverImage(c echo.Context) error { mediaID := c.Param("id") mediaUUID, err := uuid.Parse(mediaID) if err != nil { return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid media item id"}) } item, err := mh.db.GetMediaItem(c.Request().Context(), pgtype.UUID{Bytes: mediaUUID, Valid: true}) if err != nil { if err == pgx.ErrNoRows { return c.JSON(http.StatusNotFound, map[string]string{"error": "media item not found"}) } return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()}) } coverPath := item.CoverImagePath.String if coverPath == "" { return c.JSON(http.StatusNotFound, map[string]string{"error": "cover image not found"}) } // Resolve relative path to absolute filesystem path fullPath, err := mh.getFullFilePath(c.Request().Context(), item.LibraryID, coverPath) if err != nil { return c.JSON(http.StatusNotFound, map[string]string{"error": "cover image not found"}) } if _, err := os.Stat(fullPath); os.IsNotExist(err) { return c.JSON(http.StatusNotFound, map[string]string{"error": "cover image file not found"}) } ext := strings.ToLower(filepath.Ext(fullPath)) contentType := map[string]string{ ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".png": "image/png", ".gif": "image/gif", ".webp": "image/webp", }[ext] if contentType == "" { contentType = "application/octet-stream" } c.Response().Header().Set("Content-Type", contentType) c.Response().Header().Set("Cache-Control", "public, max-age=86400") return c.File(fullPath) } ``` ### File: `internal/router/media.go` **Location**: After existing media routes ```go // Cover images - authenticated protected.GET("/covers/:id", cfg.MediaHandler.GetCoverImage) ``` --- ## Phase 6: Frontend - No Changes Needed (SSR) The existing frontend code should work without modification: - **dashboard.templ**: Uses `item.CoverImagePath` directly in SSR - **dashboard.ts**: Uses `book.cover_image_path` from API response - **bookshelf.ts**: Uses `book.cover_image_path` from API response **Note**: If covers don't load initially, it's because the handler needs to resolve the URL. The frontend code doesn't need to change - the backend just needs to provide usable URLs. If the endpoint responses include a resolved URL field (like `cover_url`), update the frontend to use that field. Otherwise, the existing `/api/covers/:id` endpoint can be used. --- ## Phase 7: Backward Compatibility Handle existing absolute paths in database: ### Option A: Migration (One-time) Create a script to convert existing absolute paths to relative paths using known library folder paths. ### Option B: Runtime Resolution (No migration) Add backward compatibility in handlers: ```go func (mh *MediaHandler) getFullFilePath(ctx context.Context, libraryID pgtype.UUID, relativePath string) (string, error) { // Already absolute? Use as-is (backward compatibility) if filepath.IsAbs(relativePath) { return relativePath, nil } // Otherwise resolve as relative path return mh.libraryService.ResolveMediaPath(ctx, libraryID, relativePath) } ``` **Recommended**: Option B - no database migration needed, handles both old and new data. --- ## Phase 8: Tests ### Unit Tests #### File: `internal/handlers/media_test.go` ```go // TestGetCoverImage_ValidItem tests successful cover image retrieval func TestGetCoverImage_ValidItem(t *testing.T) { // Setup test server with mock database // Create a test cover image file // Call GetCoverImage // Verify response has correct Content-Type and status code } // TestGetCoverImage_NotFound tests 404 for non-existent media item func TestGetCoverImage_NotFound(t *testing.T) { // Call with invalid UUID // Verify 404 response } // TestGetCoverImage_NoCover tests 404 when media item has no cover func TestGetCoverImage_NoCover(t *testing.T) { // Create media item with empty cover_image_path // Verify 404 response } // TestGetFullFilePath_RelativePath tests relative path resolution func TestGetFullFilePath_RelativePath(t *testing.T) { // Setup: Create library with folder /app/uploads // Media item with file_path: "Author/Book/book.epub" // Call getFullFilePath // Verify returns: "/app/uploads/Author/Book/book.epub" } // TestGetFullFilePath_AbsolutePath tests backward compatibility func TestGetFullFilePath_AbsolutePath(t *testing.T) { // Media item with absolute file_path // Verify returns same path } ``` ### Scanner Tests #### File: `internal/services/media_scanner_test.go` ```go // TestGetRelativePath tests path conversion func TestGetRelativePath(t *testing.T) { scanner := &MediaScanner{ folders: []string{"/app/uploads", "/var/books"}, } tests := []struct { absolute string expected string }{ {"/app/uploads/Author/Book/epub", "Author/Book/epub"}, {"/var/books/manga/Naruto/vol1", "manga/Naruto/vol1"}, {"/other/path/file.pdf", "/other/path/file.pdf"}, // fallback } for _, tt := range tests { result := scanner.getRelativePath(tt.absolute) assert.Equal(t, tt.expected, result) } } ``` ### Integration Tests #### File: `cmd/server/tests/cover_file_serving_test.go` ```go package main import ( "bytes" "encoding/json" "fmt" "io" "net/http" "os" "path/filepath" "testing" "time" "github.com/stretchr/testify/require" "github.com/stretchr/testify/suite" ) ``` Note: The integration tests use `setupTestServer(s.T())` from `cmd/server/tests/test_helpers.go` as per PROJECT_GUIDELINES.md requirements. ### Bruno API Tests Create new Bruno test files for the new endpoints: #### File: `bruno/media-items/Get Cover Image.yml` ```yaml info: name: Get Cover Image type: http seq: 1 http: method: GET url: '{{base_url}}/api/covers/{{media_item_id}}' auth: none docs: |- ## Get Cover Image Retrieve the cover image for a media item. Requires authentication. **Method:** GET **Endpoint:** /api/covers/{id} **Authentication:** Bearer token required **Response:** Binary image data (JPEG, PNG, etc.) **Status Codes:** - 200: Success - returns image - 400: Invalid media item ID - 401: Unauthorized - 404: Media item not found or cover doesn't exist Note: Uses `media_item_id` from environment variables. ``` #### File: `bruno/media-items/Download Media Item.yml` (Update existing) Update the existing file to document that it now handles relative paths: ```yaml info: name: Download Media Item type: http seq: 1 http: method: GET url: '{{base_url}}/api/media-items/{{media_item_id}}/download' auth: none docs: |- ## Download Media Item Download a media item file (EPUB, PDF, CBZ, etc.) from Bookhoard server. **Method:** GET **Endpoint:** /api/media-items/{id}/download **Authentication:** Bearer token required **Path Resolution:** The handler resolves the relative file path stored in the database against the library's configured folder(s) to locate the actual file. **Backward Compatibility:** Supports both relative paths (new) and absolute paths (legacy data). **Response:** Binary file data with appropriate Content-Type header **Status Codes:** - 200: Success - returns file - 400: Invalid media item ID - 401: Unauthorized - 404: Media item not found or file doesn't exist on disk Note: Uses `media_item_id` from environment variables. ``` --- ## Phase 9: Documentation ### File: `docs/developer/api/media-items/get_cover_image.md` ```markdown --- title: Get Cover Image description: Retrieve the cover image for a media item --- # Get Cover Image Retrieve the cover image for a media item. ## Endpoint `GET /api/covers/:id` ## Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | id | string | The media item ID (UUID) | ## Headers | Header | Required | Description | |--------|----------|-------------| | Authorization | Yes | Bearer token | ## Response - **200 OK**: Cover image returned - Content-Type: `image/jpeg`, `image/png`, etc. - Cache-Control: `public, max-age=86400` - **400 Bad Request**: Invalid media item ID - **404 Not Found**: - Media item not found - No cover image configured - Cover image file not found on disk ## Example ```bash curl -H "Authorization: Bearer YOUR_TOKEN" \ http://localhost:8765/api/covers/550e8400-e29b-41d4-a716-446655440000 \ --output cover.jpg ``` ## Notes - Cover images are stored relative to their library folder - The API resolves the full path using the library's configured folder(s) - Supports backward compatibility with existing absolute paths - Images are cached for 24 hours by clients - All endpoints require authentication (JWT) ``` ### File: `docs/developer/api/media-items/download_book.md` Update existing documentation to note: - File paths are stored relative to library folders - Handler resolves path at request time - Backward compatible with existing absolute paths --- ## Summary of Changes | Phase | File | Change | |-------|------|--------| | 1 | `internal/services/media_scanner.go` | Add `getRelativePath()` function; use for both file_path and cover_path | | 2 | `internal/services/library_service.go` | Add `ResolveMediaPath()` function | | 3 | `internal/handlers/media.go` | Modify `ListMediaItems` to include resolved URLs if needed | | 4 | `internal/handlers/media.go` | Modify `DownloadBook` to resolve relative paths | | 5 | `internal/handlers/media.go` | Add `GetCoverImage` function | | 5 | `internal/router/media.go` | Add route `GET /api/covers/:id` | | 6 | Frontend files | No changes needed (verify in testing) | | 7 | Backward compatibility | Runtime resolution handles absolute paths | | 8 | `internal/handlers/media_test.go` | Add unit tests for GetCoverImage and path resolution | | 8 | `internal/services/media_scanner_test.go` | Add unit tests for getRelativePath | | 8 | `cmd/server/tests/cover_file_serving_test.go` | Add integration tests using test_helpers | | 9 | `bruno/media-items/Get Cover Image.yml` | Add Bruno API test for cover endpoint | | 9 | `bruno/media-items/EPUB Download.yml` | Update to document relative path handling | | 9 | `docs/developer/api/media-items/get_cover_image.md` | Add API documentation | --- ## Verification Steps After implementation: 1. **Test new scan**: Add a new book with cover, verify: - Database `file_path` is relative (e.g., `Author/Book/book.epub`) - Database `cover_image_path` is relative (e.g., `Author/Book/cover.jpg`) - GET `/api/media-items/:id/download` returns the file - GET `/api/covers/:id` returns the image 2. **Test existing data**: For items with absolute paths: - Downloads still work (backward compatibility) - Cover images still work (backward compatibility) 3. **Test multiple mount points**: - Library A with folder `/app/epubs` - Library B with folder `/var/manga` - Books in each resolve correctly 4. **Test frontend**: - Dashboard shows cover images - Bookshelf shows cover images - Downloads work 5. **Test mobile app** (future): - Same JWT auth works for files and covers --- ## Flexibility for Users Users can configure any mount point in docker-compose: ```yaml services: bookhoard: volumes: - ./epubs:/app/epubs # ebooks - ./manga:/var/manga # manga - ./comics:/media/comics # comics ``` The system stores relative paths, so it works with any configuration.