docs(schema): Consolidate schema initialization plans with corrections
Remove obsolete planning documents: - KOBO_IMPLEMENTATION_PLAN.md (replaced by refined plan) - SCHEMA_INITIALIZATION_PLAN.md (replaced by refined plan) Update REFINED_SCHEMA_PLAN.md with critical corrections: - Fix FNV-1a hash constant (7804706162000639061, was 582394759234) - Correct table/index counts (27 tables, 82 indexes, not 36/102) - Change approach: embed existing schema.sql (no duplication) - Add local database update step after schema changes - Add documentation requirements section This consolidates three planning documents into one accurate, actionable plan for implementing automatic schema initialization with idempotent migrations.
This commit is contained in:
@@ -1,825 +0,0 @@
|
|||||||
# 📋 **Complete Implementation Plan: Fix Kobo Production Code & Add "last-read-place" Support**
|
|
||||||
|
|
||||||
## 🎯 **Executive Summary**
|
|
||||||
|
|
||||||
**Critical Finding**: Kobo DOES send "last-read-place" bookmarks with precise position data, but Bookhoard currently ignores them (falls through switch statement). Adding this support will significantly improve Kobo user experience.
|
|
||||||
|
|
||||||
**Compliance**: All changes follow PROJECT_GUIDELINES.md with:
|
|
||||||
- No local builds (Podman/Docker only)
|
|
||||||
- No breaking changes without testing
|
|
||||||
- Step-by-step implementation (no cascading fix-ups)
|
|
||||||
- Clear git commits with detailed messages
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ⚠️ **Pre-Execution Checklist**
|
|
||||||
|
|
||||||
- [ ] Create backup branch: `git branch backup-before-kobo-comprehensive-fixes`
|
|
||||||
- [ ] Run baseline test suite: `make test-integration > baseline.txt 2>&1`
|
|
||||||
- [ ] Verify containers: `podman compose ps`
|
|
||||||
- [ ] Confirm in build mode (not local)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📊 **Phase 1: Critical Bug Fixes** (45 minutes)
|
|
||||||
|
|
||||||
### **Step 1: Fix Rate Limit Header Bug**
|
|
||||||
**File**: `/internal/middleware/device_auth.go`
|
|
||||||
**Lines**: 79, 84, 90
|
|
||||||
|
|
||||||
**Current Code**:
|
|
||||||
```go
|
|
||||||
c.Response().Header().Set("X-RateLimit-Remaining", string(rune(remaining)))
|
|
||||||
```
|
|
||||||
|
|
||||||
**Issue**: Converts int to Unicode character (60 → `<`)
|
|
||||||
|
|
||||||
**Fix**:
|
|
||||||
```go
|
|
||||||
import "strconv"
|
|
||||||
c.Response().Header().Set("X-RateLimit-Remaining", strconv.Itoa(remaining))
|
|
||||||
```
|
|
||||||
|
|
||||||
**Verification**:
|
|
||||||
- Edit file
|
|
||||||
- `make build` && `podman compose restart app`
|
|
||||||
- Test rate-limited endpoint with curl
|
|
||||||
- Verify header returns integer
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### **Step 2: Fix Type Assertion Bug**
|
|
||||||
**File**: `/internal/middleware/device_auth.go`
|
|
||||||
**Lines**: 168-171
|
|
||||||
|
|
||||||
**Current Code**:
|
|
||||||
```go
|
|
||||||
deviceID, ok := c.Get("device_id").(uuid.UUID) // Always fails!
|
|
||||||
if ok {
|
|
||||||
m.db.UpdateDeviceLastSeen(c.Request().Context(), pgDeviceID) // Never executes
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Issue**: `device_id` is `[16]byte`, cast tries `uuid.UUID`
|
|
||||||
|
|
||||||
**Fix**:
|
|
||||||
```go
|
|
||||||
deviceIDBytes, ok := c.Get("device_id").([16]byte)
|
|
||||||
if ok {
|
|
||||||
pgDeviceID := pgtype.UUID{Bytes: deviceIDBytes, Valid: true}
|
|
||||||
m.db.UpdateDeviceLastSeen(c.Request().Context(), pgDeviceID)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### **Step 3: Fix user_id Type Mismatch**
|
|
||||||
**File**: `/internal/middleware/device_auth.go`
|
|
||||||
**Line**: 105
|
|
||||||
|
|
||||||
**Analysis**: No handlers use `c.Get("user_id")`, so line is redundant
|
|
||||||
|
|
||||||
**Fix**: Remove line entirely
|
|
||||||
```go
|
|
||||||
// DELETE: c.Set("user_id", device.UserID.Bytes)
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### **Step 4: Fix Kobo Nil UUID Error Handling**
|
|
||||||
**File**: `/internal/handlers/kobo.go`
|
|
||||||
**Lines**: 402, 437, 515, 580
|
|
||||||
|
|
||||||
**Current Code**:
|
|
||||||
```go
|
|
||||||
bookhoardUUID, err, _ := h.mapContentIdToBookhoardUUID(c, contentId, deviceUUID)
|
|
||||||
if err != nil {
|
|
||||||
unlinkedBooks++
|
|
||||||
continue // Uses potentially nil bookhoardUUID!
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Fix**:
|
|
||||||
```go
|
|
||||||
bookhoardUUID, err, _ := h.mapContentIdToBookhoardUUID(c, contentId, deviceUUID)
|
|
||||||
if err != nil || bookhoardUUID == uuid.Nil {
|
|
||||||
unlinkedBooks++
|
|
||||||
continue // Skip nil UUIDs safely
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### **Step 5: Add ContentType Detection**
|
|
||||||
**File**: `/internal/handlers/kobo.go`
|
|
||||||
**Lines**: Add to ReadReadingSync struct
|
|
||||||
|
|
||||||
**Current Code**:
|
|
||||||
```go
|
|
||||||
type KoboReadingSync struct {
|
|
||||||
ContentId string `json:"ContentId"`
|
|
||||||
PercentRead float64 `json:"PercentRead"`
|
|
||||||
EntitlementId string `json:"EntitlementId"`
|
|
||||||
RemainingTimeMinutes int `json:"RemainingTimeMinutes"`
|
|
||||||
FirstReadTime string `json:"FirstReadTime,omitempty"`
|
|
||||||
LastModified string `json:"LastModified"`
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Fix**:
|
|
||||||
```go
|
|
||||||
type KoboReadingSync struct {
|
|
||||||
// ... existing fields ...
|
|
||||||
ContentType string `json:"ContentType,omitempty"` // NEW
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Update Initialization handler**:
|
|
||||||
```go
|
|
||||||
contentType := "6" // Default EPUB
|
|
||||||
if strings.Contains(mediaItem.MimeType.String, "pdf") {
|
|
||||||
contentType = "5" // PDF
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📊 **Phase 2: Kobo Enhancement** (30 minutes)
|
|
||||||
|
|
||||||
### **Step 6: Add "last-read-place" Support**
|
|
||||||
**File**: `/internal/handlers/kobo.go`
|
|
||||||
**Lines**: Add case to existing switch statement
|
|
||||||
|
|
||||||
**Current Implementation** (lines 445-468):
|
|
||||||
```go
|
|
||||||
switch bookmarkSync.BookmarkType {
|
|
||||||
case "annotation":
|
|
||||||
if bookmarkSync.BookmarkText != "" {
|
|
||||||
// Create highlight
|
|
||||||
h.db.CreateMediaHighlight(...)
|
|
||||||
}
|
|
||||||
case "bookmark":
|
|
||||||
if bookmarkSync.BookmarkText != "" {
|
|
||||||
// Create note
|
|
||||||
h.db.CreateMediaNote(...)
|
|
||||||
}
|
|
||||||
// "last-read-place" - FALLS THROUGH, IGNORED
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Fix**:
|
|
||||||
```go
|
|
||||||
switch bookmarkSync.BookmarkType {
|
|
||||||
case "annotation":
|
|
||||||
if bookmarkSync.BookmarkText != "" {
|
|
||||||
// Create highlight
|
|
||||||
h.db.CreateMediaHighlight(...)
|
|
||||||
}
|
|
||||||
case "bookmark":
|
|
||||||
if bookmarkSync.BookmarkText != "" {
|
|
||||||
// Create note
|
|
||||||
h.db.CreateMediaNote(...)
|
|
||||||
}
|
|
||||||
case "last-read-place":
|
|
||||||
// Extract precise position from BookmarkId
|
|
||||||
if bookmarkSync.BookmarkId != "" {
|
|
||||||
var epubcfi, chapter string
|
|
||||||
|
|
||||||
// Parse EPUB CFI format if present
|
|
||||||
if strings.HasPrefix(bookmarkSync.BookmarkId, "epubcfi(") {
|
|
||||||
epubcfi = strings.TrimPrefix(bookmarkSync.BookmarkId, "epubcfi(")
|
|
||||||
epubcfi = strings.TrimSuffix(epubcfi, ")")
|
|
||||||
}
|
|
||||||
|
|
||||||
// Store position data in reading_progress
|
|
||||||
_, err = h.db.UpdateUniversalProgress(c.Request().Context(), database.UpdateUniversalProgressParams{
|
|
||||||
MediaItemID: pgMediaUUID,
|
|
||||||
UserID: pgUserID,
|
|
||||||
Epubcfi: pgtype.Text{String: epubcfi, Valid: true},
|
|
||||||
Chapter: pgtype.Int4{Int32: int32(chapter), Valid: true},
|
|
||||||
ChapterProgress: pgtype.Float8{Float64: 0.5, Valid: true},
|
|
||||||
DeviceSyncData: pgtype.JSONB{
|
|
||||||
Bytes: []byte(fmt.Sprintf(`{"kobo_bookmark_id": "%s", "hidden": %v}`,
|
|
||||||
bookmarkSync.BookmarkId, bookmarkSync.Hidden)),
|
|
||||||
Valid: true,
|
|
||||||
},
|
|
||||||
LastSyncDevice: pgtype.Text{String: "kobo", Valid: true},
|
|
||||||
LastSyncSource: pgtype.Text{String: "kobo", Valid: true},
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
log.Printf("Failed to store last-read-place: %v", err)
|
|
||||||
}
|
|
||||||
bookmarksSynced++
|
|
||||||
}
|
|
||||||
default:
|
|
||||||
log.Printf("Unknown bookmark type: %s", bookmarkSync.BookmarkType)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📊 **Phase 3: Test Implementation & Updates** (45 minutes)
|
|
||||||
|
|
||||||
### **Step 7: Add Debug Logging**
|
|
||||||
**File**: `/cmd/server/tests/kobo_test.go`
|
|
||||||
|
|
||||||
**Add before each test**:
|
|
||||||
```go
|
|
||||||
log.Printf("[DEBUG] Kobo test setup: Creating device and media")
|
|
||||||
log.Printf("[DEBUG] Kobo device: ID=%s, Token=%s", device.ID, device.AuthToken)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Add to Markup function**:
|
|
||||||
```go
|
|
||||||
log.Printf("[DEBUG] Processing Kobo bookmark: Type=%s, Text=%q, Hidden=%v, ID=%s",
|
|
||||||
bookmarkSync.BookmarkType,
|
|
||||||
bookmarkSync.BookmarkText,
|
|
||||||
bookmarkSync.Hidden,
|
|
||||||
bookmarkSync.BookmarkId)
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### **Step 8: Fix TestKoboInitialization**
|
|
||||||
**File**: `/cmd/server/tests/kobo_test.go`
|
|
||||||
**Lines**: 27-38
|
|
||||||
|
|
||||||
**Current Issues**:
|
|
||||||
1. No device created
|
|
||||||
2. Wrong route path (`/test-token/` segment)
|
|
||||||
3. Hardcoded token doesn't exist
|
|
||||||
|
|
||||||
**Fix**:
|
|
||||||
```go
|
|
||||||
func TestKoboInitialization(t *testing.T) {
|
|
||||||
if testing.Short() {
|
|
||||||
t.Skip("Skipping integration test in short mode")
|
|
||||||
}
|
|
||||||
|
|
||||||
ts, db, _ := setupTestServer(t)
|
|
||||||
defer closeTestServer(t, ts, db)
|
|
||||||
|
|
||||||
token := loginTestUser(t, ts, db)
|
|
||||||
_ = createTestMediaItemID(t, ts, token)
|
|
||||||
|
|
||||||
// Create Kobo device for proper authentication
|
|
||||||
device := setup.CreateDevice(t, "Test Kobo", "kobo", "kobo-clara-test")
|
|
||||||
|
|
||||||
t.Run("successful initialization", func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("GET", ts.URL+"/api/sync/kobo/v1/initialization", nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+device.AuthToken)
|
|
||||||
req.Header.Set("x-kobo-device", fmt.Sprintf(`{"DeviceId":"%s","Model":"Kobo Clara","SerialNumber":"%s"}`,
|
|
||||||
device.ID.String(), device.Identifier))
|
|
||||||
|
|
||||||
client := &http.Client{}
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### **Step 9: Fix TestKoboLibrarySync**
|
|
||||||
**File**: `/cmd/server/tests/kobo_test.go`
|
|
||||||
**Lines**: 51-62
|
|
||||||
|
|
||||||
**Fixes**: Same pattern as TestKoboInitialization
|
|
||||||
```go
|
|
||||||
func TestKoboLibrarySync(t *testing.T) {
|
|
||||||
if testing.Short() {
|
|
||||||
t.Skip("Skipping integration test in short mode")
|
|
||||||
}
|
|
||||||
|
|
||||||
ts, db, _ := setupTestServer(t)
|
|
||||||
defer closeTestServer(t, ts, db)
|
|
||||||
|
|
||||||
token := loginTestUser(t, ts, db)
|
|
||||||
_ = createTestMediaItemID(t, ts, token)
|
|
||||||
|
|
||||||
// Create Kobo device
|
|
||||||
device := setup.CreateDevice(t, "Test Kobo", "kobo", "kobo-clara-test")
|
|
||||||
|
|
||||||
t.Run("successful library sync", func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("GET", ts.URL+"/api/sync/kobo/v1/initialization", nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+device.AuthToken)
|
|
||||||
req.Header.Set("x-kobo-device", fmt.Sprintf(`{"DeviceId":"%s","Model":"Kobo Clara","SerialNumber":"%s"}`,
|
|
||||||
device.ID.String(), device.Identifier))
|
|
||||||
|
|
||||||
client := &http.Client{}
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### **Step 10: Fix TestKoboMarkupSync**
|
|
||||||
**File**: `/cmd/server/tests/kobo_test.go`
|
|
||||||
**Lines**: 74-118
|
|
||||||
|
|
||||||
**Add device creation**:
|
|
||||||
```go
|
|
||||||
ts, db, _ := setupTestServer(t)
|
|
||||||
defer closeTestServer(t, ts, db)
|
|
||||||
|
|
||||||
token := loginTestUser(t, ts, db)
|
|
||||||
mediaItemID := createTestMediaItemID(t, ts, token)
|
|
||||||
|
|
||||||
// Create Kobo device
|
|
||||||
device := setup.CreateDevice(t, "Test Kobo", "kobo", "kobo-clara-test")
|
|
||||||
|
|
||||||
t.Run("successful markup sync with annotations and bookmarks", func(t *testing.T) {
|
|
||||||
reqBody := map[string]interface{}{
|
|
||||||
"ReadingSync": []map[string]interface{}{
|
|
||||||
{
|
|
||||||
"ContentId": mediaItemID,
|
|
||||||
"PercentRead": 45.6,
|
|
||||||
"EntitlementId": "ent-123",
|
|
||||||
"RemainingTimeMinutes": 120,
|
|
||||||
"LastModified": "2026-01-30T20:00:00Z",
|
|
||||||
},
|
|
||||||
},
|
|
||||||
"BookmarkSync": []map[string]interface{}{
|
|
||||||
{
|
|
||||||
"BookmarkId": "bookmark-1",
|
|
||||||
"ContentId": mediaItemID,
|
|
||||||
"BookmarkText": "This is highlighted text",
|
|
||||||
"BookmarkType": "annotation",
|
|
||||||
"BookmarkTitle": "Chapter 3",
|
|
||||||
"DateCreated": "2026-01-30T19:55:00Z",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"BookmarkId": "bookmark-2",
|
|
||||||
"ContentId": mediaItemID,
|
|
||||||
"BookmarkText": "This is my note about book",
|
|
||||||
"BookmarkType": "bookmark",
|
|
||||||
"DateCreated": "2026-01-30T19:55:00Z",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"BookmarkId": "epubcfi(/6/4[chap1]!/4/2/1:0)",
|
|
||||||
"ContentId": mediaItemID,
|
|
||||||
"BookmarkType": "last-read-place",
|
|
||||||
"Hidden": true,
|
|
||||||
"DateCreated": "2026-01-30T19:55:00Z",
|
|
||||||
}, // NEW: Test last-read-place bookmark!
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
body, _ := json.Marshal(reqBody)
|
|
||||||
req, _ := http.NewRequest("POST", ts.URL+"/api/sync/kobo/markup", bytes.NewReader(body))
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
req.Header.Set("Authorization", "Bearer "+device.AuthToken)
|
|
||||||
req.Header.Set("x-kobo-device", fmt.Sprintf(`{"DeviceId":"%s","Model":"Kobo Clara","SerialNumber":"%s"}`,
|
|
||||||
device.ID.String(), device.Identifier))
|
|
||||||
|
|
||||||
client := &http.Client{}
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode)
|
|
||||||
var result map[string]interface{}
|
|
||||||
json.NewDecoder(resp.Body).Decode(&result)
|
|
||||||
assert.Contains(t, result, "Status")
|
|
||||||
|
|
||||||
// NEW: Verify last-read-place was processed
|
|
||||||
assert.Contains(t, result, "bookmarks_synced")
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### **Step 11: Fix TestKoboBookmarkSync**
|
|
||||||
**File**: `/cmd/server/tests/kobo_test.go`
|
|
||||||
**Lines**: 130-159
|
|
||||||
|
|
||||||
**Add device creation and last-read-place test**:
|
|
||||||
```go
|
|
||||||
t.Run("successful bookmark sync with last-read-place", func(t *testing.T) {
|
|
||||||
reqBody := map[string]interface{}{
|
|
||||||
"BookmarkSync": []map[string]interface{}{
|
|
||||||
{
|
|
||||||
"BookmarkId": "bookmark-3",
|
|
||||||
"ContentId": mediaItemID,
|
|
||||||
"BookmarkText": "Important note about book",
|
|
||||||
"BookmarkType": "bookmark",
|
|
||||||
"DateCreated": "2026-01-30T19:55:00Z",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"BookmarkId": "epubcfi(/6/4[chap1]!/4/2/1:156)",
|
|
||||||
"ContentId": mediaItemID,
|
|
||||||
"BookmarkType": "last-read-place",
|
|
||||||
"Hidden": true,
|
|
||||||
"DateCreated": "2026-01-30T19:55:00Z",
|
|
||||||
}, // NEW: Test precise position bookmark
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
body, _ := json.Marshal(reqBody)
|
|
||||||
req, _ := http.NewRequest("POST", ts.URL+"/api/sync/kobo/bookmark", bytes.NewReader(body))
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
req.Header.Set("Authorization", "Bearer "+device.AuthToken)
|
|
||||||
req.Header.Set("x-kobo-device", fmt.Sprintf(`{"DeviceId":"%s","Model":"Kobo Clara","SerialNumber":"%s"}`,
|
|
||||||
device.ID.String(), device.Identifier))
|
|
||||||
|
|
||||||
client := &http.Client{}
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode)
|
|
||||||
var result map[string]interface{}
|
|
||||||
json.NewDecoder(resp.Body).Decode(&result)
|
|
||||||
assert.Contains(t, result, "bookmarks_synced")
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### **Step 12: Fix TestKoboAnalyticsGettests**
|
|
||||||
**File**: `/cmd/server/tests/kobo_test.go`
|
|
||||||
**Lines**: 172-198
|
|
||||||
|
|
||||||
**Add device creation**:
|
|
||||||
```go
|
|
||||||
ts, db, _ := setupTestServer(t)
|
|
||||||
defer closeTestServer(t, ts, db)
|
|
||||||
|
|
||||||
token := loginTestUser(t, ts, db)
|
|
||||||
mediaItemID := createTestMediaItemID(t, ts, token)
|
|
||||||
|
|
||||||
// Create Kobo device
|
|
||||||
device := setup.CreateDevice(t, "Test Kobo", "kobo", "kobo-clara-test")
|
|
||||||
|
|
||||||
t.Run("successful analytics tests", func(t *testing.T) {
|
|
||||||
reqBody := map[string]interface{}{
|
|
||||||
"meta": map[string]string{
|
|
||||||
"name": "Kobo Analytics Tests",
|
|
||||||
},
|
|
||||||
"ContentId": mediaItemID,
|
|
||||||
"ReadingEvent": "Reading",
|
|
||||||
"RemainingTimeMin": 180,
|
|
||||||
"PercentRead": 67.8,
|
|
||||||
}
|
|
||||||
|
|
||||||
body, _ := json.Marshal(reqBody)
|
|
||||||
req, _ := http.NewRequest("POST", ts.URL+"/api/sync/kobo/v1/analytics/gettests", bytes.NewReader(body))
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
req.Header.Set("Authorization", "Bearer "+device.AuthToken)
|
|
||||||
req.Header.Set("x-kobo-device", fmt.Sprintf(`{"DeviceId":"%s","Model":"Kobo Clara","SerialNumber":"%s"}`,
|
|
||||||
device.ID.String(), device.Identifier))
|
|
||||||
|
|
||||||
client := &http.Client{}
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode)
|
|
||||||
var result map[string]interface{}
|
|
||||||
json.NewDecoder(resp.Body).Decode(&result)
|
|
||||||
assert.Contains(t, result, "Status")
|
|
||||||
})
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📊 **Phase 4: Verification** (30 minutes)
|
|
||||||
|
|
||||||
### **Step 13: Critical Bug Verification**
|
|
||||||
```bash
|
|
||||||
# Test rate limit headers
|
|
||||||
curl -H "Authorization: Bearer dev_test123" \
|
|
||||||
-H "x-kobo-device: {\"DeviceId\":\"test\",\"Model\":\"test\"}" \
|
|
||||||
http://localhost:8765/api/sync/kobo/markup | \
|
|
||||||
-i "X-RateLimit-Remaining"
|
|
||||||
|
|
||||||
# Test device last_seen updates
|
|
||||||
podman exec bookhoard_db psql -U postgres -d bookhoard -c \
|
|
||||||
"SELECT device_name, last_seen FROM devices WHERE device_type = 'kobo' ORDER BY last_seen DESC LIMIT 5"
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Step 14: Full Test Suite**
|
|
||||||
```bash
|
|
||||||
make test-integration > test-results.txt 2>&1
|
|
||||||
grep -E "(RUN|PASS|FAIL).*TestKobo" test-results.txt
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Step 15: Debug Log Analysis**
|
|
||||||
```bash
|
|
||||||
podman compose logs app | grep -i "last-read-place" | tail -10
|
|
||||||
podman compose logs app | grep -i "kobo.*bookmark" | tail -20
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📝 **Documentation Updates Required**
|
|
||||||
|
|
||||||
### **1. API Documentation Updates**
|
|
||||||
|
|
||||||
**File**: `/docs/developer/api/kobo/markup_sync.md`
|
|
||||||
|
|
||||||
**Add to "Bookmark Types" section**:
|
|
||||||
```markdown
|
|
||||||
#### Bookmark Types
|
|
||||||
|
|
||||||
| Type | Description | Storage |
|
|
||||||
|-------|-------------|----------|
|
|
||||||
| "annotation" | Highlighted text with optional notes | `media_highlights` table |
|
|
||||||
| "bookmark" | User-created bookmarks with notes | `media_notes` table |
|
|
||||||
| "last-read-place" | Auto-generated position marker | `reading_progress` table (position fields) |
|
|
||||||
```
|
|
||||||
|
|
||||||
**Add to "Response Format" section**:
|
|
||||||
```markdown
|
|
||||||
#### Response Example
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"bookmarks_synced": 3,
|
|
||||||
"highlights_synced": 1,
|
|
||||||
"notes_synced": 1,
|
|
||||||
"positions_updated": 1
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### **File**: `/docs/developer/api/kobo/bookmark_sync.md`
|
|
||||||
|
|
||||||
**Add similar Bookmark Type documentation**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### **2. Bruno API Test Updates**
|
|
||||||
|
|
||||||
**File**: `/bruno/devices/Kobo Bookmark Sync.bru`
|
|
||||||
|
|
||||||
**Add new request example**:
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"name": "Kobo Bookmark Sync with Last-Read-Place",
|
|
||||||
"method": "POST",
|
|
||||||
"url": "{{baseUrl}}/api/sync/kobo/bookmark",
|
|
||||||
"headers": {
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
"Authorization": "Bearer {{deviceAuthToken}}",
|
|
||||||
"x-kobo-device": "{{koboDeviceHeader}}"
|
|
||||||
},
|
|
||||||
"body": {
|
|
||||||
"BookmarkSync": [
|
|
||||||
{
|
|
||||||
"BookmarkId": "epubcfi(/6/4[chap1]!/4/2/1:156)",
|
|
||||||
"ContentId": "{{mediaItemId}}",
|
|
||||||
"BookmarkType": "last-read-place",
|
|
||||||
"Hidden": true,
|
|
||||||
"DateCreated": "2026-01-30T19:55:00Z"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"tests": {
|
|
||||||
"shouldProcessPositionBookmarks": {
|
|
||||||
"status": 200,
|
|
||||||
"body": {
|
|
||||||
"bookmarks_synced": 1
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### **3. New Bruno Test File**
|
|
||||||
|
|
||||||
**Create**: `/bruno/devices/Kobo Last-Read-Place Sync.bru`
|
|
||||||
|
|
||||||
**Content**:
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"name": "Kobo Last-Read-Place Position Sync",
|
|
||||||
"method": "POST",
|
|
||||||
"url": "{{baseUrl}}/api/sync/kobo/markup",
|
|
||||||
"headers": {
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
"Authorization": "Bearer {{deviceAuthToken}}",
|
|
||||||
"x-kobo-device": "{{koboDeviceHeader}}"
|
|
||||||
},
|
|
||||||
"body": {
|
|
||||||
"ReadingSync": [
|
|
||||||
{
|
|
||||||
"ContentId": "{{mediaItemId}}",
|
|
||||||
"PercentRead": 45.6,
|
|
||||||
"LastModified": "2026-01-30T20:00:00Z"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"BookmarkSync": [
|
|
||||||
{
|
|
||||||
"BookmarkId": "epubcfi(/6/4[chap1]!/4/2/1:0)",
|
|
||||||
"ContentId": "{{mediaItemId}}",
|
|
||||||
"BookmarkType": "last-read-place",
|
|
||||||
"Hidden": true,
|
|
||||||
"DateCreated": "2026-01-30T19:55:00Z"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"tests": {
|
|
||||||
"shouldSyncPrecisePosition": {
|
|
||||||
"status": 200,
|
|
||||||
"body": {
|
|
||||||
"bookmarks_synced": 1,
|
|
||||||
"positions_updated": 1
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"checkProgressUpdate": {
|
|
||||||
"status": 200,
|
|
||||||
"body": {
|
|
||||||
"percentage": 0.456,
|
|
||||||
"epubcfi": "epubcfi(/6/4[chap1]!/4/2/1:0)"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📝 **Phase 5: Git Commits** (15 minutes)
|
|
||||||
|
|
||||||
### **Commit 1**: Fix device auth string conversion
|
|
||||||
```
|
|
||||||
fix(middleware): Correct rate limit header type conversion
|
|
||||||
|
|
||||||
- Fix string(rune(remaining)) to strconv.Itoa(remaining) in device_auth.go
|
|
||||||
- Prevents garbage characters in X-RateLimit-Remaining header
|
|
||||||
- No functionality changes, only fixes broken headers
|
|
||||||
|
|
||||||
Testing: Verified with curl that headers return proper integers
|
|
||||||
Fixes: #XXX
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Commit 2**: Fix device auth type assertion
|
|
||||||
```
|
|
||||||
fix(middleware): Enable device last_seen timestamp updates
|
|
||||||
|
|
||||||
- Fix type assertion from uuid.UUID to [16]byte in UpdateDeviceLastSeen
|
|
||||||
- UpdateDeviceLastSeen now executes correctly after device authentication
|
|
||||||
- Devices will now track last activity timestamp for conflict detection
|
|
||||||
|
|
||||||
Testing: Confirmed last_seen field updates in database after device requests
|
|
||||||
Fixes: #XXX
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Commit 3**: Remove redundant user_id context
|
|
||||||
```
|
|
||||||
fix(middleware): Remove redundant user_id context setting
|
|
||||||
|
|
||||||
- Removed redundant c.Set("user_id", device.UserID.Bytes) from device auth
|
|
||||||
- Device authentication provides device context, no need for JWT user_id field
|
|
||||||
- No handlers use device-sourced user_id, simplifying authentication flow
|
|
||||||
|
|
||||||
Testing: Verified KOReader and Kobo sync still work correctly
|
|
||||||
Fixes: #XXX
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Commit 4**: Fix Kobo nil UUID handling
|
|
||||||
```
|
|
||||||
fix(handlers): Add nil UUID checks in Kobo sync handlers
|
|
||||||
|
|
||||||
- Check for uuid.Nil after mapContentIdToBookhoardUUID in multiple locations
|
|
||||||
- Skip database operations with invalid UUIDs to prevent errors
|
|
||||||
- Improves error handling robustness across all Kobo sync functions
|
|
||||||
|
|
||||||
Testing: Confirmed nil UUIDs are properly skipped without database errors
|
|
||||||
Fixes: #XXX
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Commit 5**: Add ContentType detection for Kobo
|
|
||||||
```
|
|
||||||
feat(handlers): Add ContentType detection for Kobo EPUB/PDF sync
|
|
||||||
|
|
||||||
- Add ContentType field to KoboReadingSync struct for dynamic content type
|
|
||||||
- Map PDF mime types to ContentType "5" for proper device rendering
|
|
||||||
- Maintains backward compatibility with existing EPUB default "6"
|
|
||||||
|
|
||||||
Testing: Verified EPUBs return "6", PDFs return "5" in responses
|
|
||||||
Fixes: #XXX
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Commit 6**: Add "last-read-place" bookmark support for Kobo
|
|
||||||
```
|
|
||||||
feat(handlers): Add Kobo "last-read-place" bookmark support
|
|
||||||
|
|
||||||
- Add handling for "last-read-place" bookmark type in Kobo bookmark sync
|
|
||||||
- Extract precise EPUB CFI position from BookmarkId field
|
|
||||||
- Update reading_progress table with exact location and chapter data
|
|
||||||
- Set Hidden=true to exclude position bookmarks from UI lists
|
|
||||||
- Enables Kobo users to resume reading at precise paragraph location
|
|
||||||
|
|
||||||
Testing: Confirmed position data stored in reading_progress.epubcfi field
|
|
||||||
Fixes: #XXX
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Commit 7**: Update Kobo API documentation
|
|
||||||
```
|
|
||||||
docs(kobo): Update bookmark sync API documentation
|
|
||||||
|
|
||||||
- Document "last-read-place" bookmark type in bookmark sync endpoints
|
|
||||||
- Add examples of position-based bookmarks vs user annotations
|
|
||||||
- Update response format to include position update confirmation
|
|
||||||
- Clarify Hidden flag usage for auto-generated bookmarks
|
|
||||||
|
|
||||||
Fixes: #XXX
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Commit 8**: Update Kobo Bruno API tests
|
|
||||||
```
|
|
||||||
test(bruno): Add comprehensive Kobo bookmark sync examples
|
|
||||||
|
|
||||||
- Add example for "last-read-place" bookmark sync in Kobo Bookmark Sync.bru
|
|
||||||
- Create new test file Kobo Last-Read-Place Sync.bru for position validation
|
|
||||||
- Include device authentication headers and proper request structure
|
|
||||||
- Add tests for position bookmark processing and response validation
|
|
||||||
|
|
||||||
Testing: All Bruno examples work with updated API
|
|
||||||
Fixes: #XXX
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Commit 9**: Fix Kobo integration tests
|
|
||||||
```
|
|
||||||
test(kobo): Fix failing Kobo integration tests
|
|
||||||
|
|
||||||
- Create Kobo devices using CreateDevice helper for proper authentication
|
|
||||||
- Use device auth tokens instead of JWT/hardcoded tokens
|
|
||||||
- Fix route path in TestKoboInitialization (remove /test-token/ segment)
|
|
||||||
- Add Authorization: Bearer <device_token> headers to all Kobo requests
|
|
||||||
- Add x-kobo-device headers with proper device metadata
|
|
||||||
- Add test case for "last-read-place" bookmark processing
|
|
||||||
- All 5 Kobo tests now pass: TestKoboInitialization, TestKoboLibrarySync,
|
|
||||||
TestKoboMarkupSync, TestKoboBookmarkSync, TestKoboAnalyticsGettests
|
|
||||||
|
|
||||||
Testing: make test-integration shows all Kobo tests PASS
|
|
||||||
Fixes: #XXX
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ✅ **Success Criteria**
|
|
||||||
|
|
||||||
### **Phase 1 Success**:
|
|
||||||
- [ ] Rate limit headers show correct integers (verified with curl)
|
|
||||||
- [ ] Device last_seen timestamps update in database
|
|
||||||
- [ ] No panics in error scenarios
|
|
||||||
- [ ] Kobo nil UUIDs properly handled
|
|
||||||
- [ ] ContentType varies by mime type (EPUB=6, PDF=5)
|
|
||||||
- [ ] No regressions in existing tests
|
|
||||||
|
|
||||||
### **Phase 2 Success**:
|
|
||||||
- [ ] "last-read-place" bookmarks processed and stored in reading_progress
|
|
||||||
- [ ] EPUB CFI position data extracted correctly
|
|
||||||
- [ ] Position bookmarks excluded from UI (Hidden=true)
|
|
||||||
- [ ] Debug logs confirm bookmark processing
|
|
||||||
- [ ] All 5 Kobo tests pass
|
|
||||||
- [ ] Tests include realistic "last-read-place" payloads
|
|
||||||
|
|
||||||
### **Phase 3 Success**:
|
|
||||||
- [ ] Full integration test suite runs successfully
|
|
||||||
- [ ] Git history shows clean, logical commits
|
|
||||||
- [ ] Documentation updated and renders at /docs endpoint
|
|
||||||
- [ ] Bruno tests include new functionality
|
|
||||||
- [ ] Real-world Kobo device behavior patterns supported
|
|
||||||
|
|
||||||
### **Phase 4 Success**:
|
|
||||||
- [ ] All 7 commits are independent and can be reverted if needed
|
|
||||||
- [ ] Each commit builds and tests successfully
|
|
||||||
- [ ] No critical functionality was broken
|
|
||||||
- [ ] Git diff shows only intended changes
|
|
||||||
- [ ] All modified files compile successfully
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚀 **Total Estimated Time: ~3 hours**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎯 **Ready to Execute**
|
|
||||||
|
|
||||||
This plan is **fully compliant** with PROJECT_GUIDELINES.md:
|
|
||||||
- Uses Podman/Docker for all builds
|
|
||||||
- Follows KISS/DRY/YAGNI principles
|
|
||||||
- No breaking changes without verification
|
|
||||||
- Step-by-step implementation with testing
|
|
||||||
- Clear, logical git commits
|
|
||||||
- Proper error recovery procedures
|
|
||||||
- Documentation updates included
|
|
||||||
|
|
||||||
**All 9 commits are independent and can be reverted if needed.**
|
|
||||||
|
|
||||||
**Proceed with implementation?**
|
|
||||||
+83
-51
@@ -16,11 +16,15 @@
|
|||||||
|
|
||||||
## 📊 Technical Decisions
|
## 📊 Technical Decisions
|
||||||
|
|
||||||
### 1. File Structure: Single Cohesive Package
|
### 1. File Structure: Single Source of Truth
|
||||||
|
|
||||||
**Decision:** Single `internal/database/schema/schema.go` file containing all schema logic.
|
**Decision:** Single `internal/database/schema.go` file referencing existing `database/schema/schema.sql`.
|
||||||
|
|
||||||
**Rationale:** Follows Go ecosystem standards and PostgreSQL/pgx patterns.
|
**Rationale:**
|
||||||
|
- **No duplication**: Single schema file remains the source of truth
|
||||||
|
- **Clear separation**: `/database/` for schema definitions, `/internal/database/` for Go logic
|
||||||
|
- **Maintainability**: Only one schema file to maintain
|
||||||
|
- **Follows existing patterns**: Database definitions stay where they belong
|
||||||
|
|
||||||
### 2. Function Location: All in One Package
|
### 2. Function Location: All in One Package
|
||||||
|
|
||||||
@@ -71,44 +75,69 @@
|
|||||||
## 📁 Implementation File Structure
|
## 📁 Implementation File Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
internal/database/schema/
|
internal/database/
|
||||||
├── schema.go # Main initialization logic
|
├── schema.go # Main initialization logic
|
||||||
├── schema.sql # Embedded idempotent schema copy
|
├── db.go # Existing sqlc-generated code
|
||||||
└── verification.go # REMOVED - consolidated into schema.go
|
├── models.go # Existing sqlc-generated models
|
||||||
|
├── queries.sql.go # Existing sqlc-generated queries
|
||||||
|
└── ...
|
||||||
|
|
||||||
|
database/
|
||||||
|
└── schema/
|
||||||
|
└── schema.sql # Existing single source of truth schema
|
||||||
```
|
```
|
||||||
|
|
||||||
**Single Responsibility:** `schema.go` handles all database initialization logic.
|
**Single Responsibility:** `schema.go` handles all database initialization logic, referencing the existing schema file.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔧 Implementation Plan
|
## 🔧 Implementation Plan
|
||||||
|
|
||||||
### Phase 1: Cleanup (Start Fresh)
|
### Phase 1: Schema File Verification
|
||||||
|
|
||||||
#### Step 1.1: Remove Duplicate Files
|
#### Step 1.1: Ensure Schema is Idempotent
|
||||||
```bash
|
```bash
|
||||||
rm -f /home/nymusicman/Code/bookhoard/internal/database/schema/verification.go
|
# Convert remaining statements to idempotent form
|
||||||
|
# Current state: 9/27 tables, 20/82 indexes, 1/3 inserts are idempotent
|
||||||
|
|
||||||
|
# Tables needing IF NOT EXISTS (18):
|
||||||
|
# - library_types, users, system_settings, refresh_tokens, libraries
|
||||||
|
# - library_folders, library_visibility, media_items, reading_progress
|
||||||
|
# - media_ratings, media_notes, media_highlights, devices, sync_queue
|
||||||
|
# - sync_conflicts, kobo_shelves, kobo_entitlements, reading_history
|
||||||
|
|
||||||
|
# Indexes: Convert remaining 62 CREATE INDEX to CREATE INDEX IF NOT EXISTS
|
||||||
|
|
||||||
|
# Inserts needing ON CONFLICT (2):
|
||||||
|
# - Line 14: library_types (name)
|
||||||
|
# - Line 44: system_settings (setting_key)
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Step 1.2: Clean Up Schema File
|
#### Step 1.2: Verify Schema Syntax
|
||||||
```bash
|
```bash
|
||||||
# Ensure schema.sql is fully idempotent
|
# Test schema.sql syntax validity
|
||||||
grep "CREATE TABLE IF NOT EXISTS" database/schema/schema.sql | wc -l # Should be 36
|
psql -h localhost -U postgres -d postgres -f database/schema/schema.sql --echo-errors --quiet
|
||||||
grep "CREATE INDEX IF NOT EXISTS" database/schema/schema.sql | wc -l # Should be 102
|
```
|
||||||
grep "ON CONFLICT" database/schema/schema.sql | wc -l # Should be 3
|
|
||||||
|
#### Step 1.3: Update Local Database
|
||||||
|
```bash
|
||||||
|
# CRITICAL: This is pre-production, update local DB after schema.sql changes
|
||||||
|
podman compose down -v # WARNING: loses all data
|
||||||
|
podman compose up -d
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### Phase 2: Create Single Schema Package
|
### Phase 2: Create Schema Initialization Logic
|
||||||
|
|
||||||
#### Step 2.1: Create `internal/database/schema/schema.go`
|
#### Step 2.1: Create `internal/database/schema.go`
|
||||||
|
|
||||||
**Implementation Strategy:**
|
**Implementation Strategy:**
|
||||||
```go
|
```go
|
||||||
package schema
|
package database
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"bufio"
|
||||||
"context"
|
"context"
|
||||||
"embed"
|
"embed"
|
||||||
"fmt"
|
"fmt"
|
||||||
@@ -121,13 +150,13 @@ import (
|
|||||||
"github.com/jackc/pgx/v5/pgxpool"
|
"github.com/jackc/pgx/v5/pgxpool"
|
||||||
)
|
)
|
||||||
|
|
||||||
//go:embed schema.sql
|
//go:embed ../../database/schema/schema.sql
|
||||||
var SchemaFile string
|
var SchemaFile string
|
||||||
|
|
||||||
const (
|
const (
|
||||||
// Pre-computed FNV-1a hash of "bookhoard:schema:init"
|
// Pre-computed FNV-1a hash of "bookhoard:schema:init"
|
||||||
// Generated using: generateLockID("bookhoard:schema:init") = 582394759234
|
// Generated using: generateLockID("bookhoard:schema:init")
|
||||||
schemaInitLockID = 582394759234
|
schemaInitLockID = 7804706162000639061
|
||||||
)
|
)
|
||||||
|
|
||||||
// Hash calculation function (for reference/testing)
|
// Hash calculation function (for reference/testing)
|
||||||
@@ -301,15 +330,14 @@ func Initialize(ctx context.Context, db *pgxpool.Pool) error {
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Step 2.2: Copy schema.sql to Package
|
#### Step 2.2: Verify Embed Path
|
||||||
```bash
|
```bash
|
||||||
cp database/schema/schema.sql internal/database/schema/schema.sql
|
# Test that the embed path resolves correctly
|
||||||
```
|
cd /home/nymusicman/Code/bookhoard/internal/database
|
||||||
|
go test -c -o /tmp/test_embed .
|
||||||
**Verification:**
|
# Or simply verify the relative path exists:
|
||||||
```bash
|
ls -la ../../database/schema/schema.sql
|
||||||
diff database/schema/schema.sql internal/database/schema/schema.sql
|
# Should show the file exists at the correct relative location
|
||||||
# Should produce no output
|
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -324,7 +352,7 @@ diff database/schema/schema.sql internal/database/schema/schema.sql
|
|||||||
```go
|
```go
|
||||||
import (
|
import (
|
||||||
// ... existing imports ...
|
// ... existing imports ...
|
||||||
"bookhoard/internal/database/schema"
|
"bookhoard/internal/database"
|
||||||
)
|
)
|
||||||
|
|
||||||
func main() {
|
func main() {
|
||||||
@@ -341,7 +369,7 @@ func main() {
|
|||||||
// ===== NEW: Schema Initialization =====
|
// ===== NEW: Schema Initialization =====
|
||||||
log.Println("🔧 Ensuring database schema is initialized...")
|
log.Println("🔧 Ensuring database schema is initialized...")
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
if err := schema.Initialize(ctx, dbPool); err != nil {
|
if err := database.Initialize(ctx, dbPool); err != nil {
|
||||||
log.Fatal("❌ Database schema initialization failed:", err)
|
log.Fatal("❌ Database schema initialization failed:", err)
|
||||||
}
|
}
|
||||||
log.Println("✅ Database schema initialized and verified, starting server...")
|
log.Println("✅ Database schema initialized and verified, starting server...")
|
||||||
@@ -358,25 +386,27 @@ func main() {
|
|||||||
### Before Declaring Complete
|
### Before Declaring Complete
|
||||||
|
|
||||||
**Phase 1 Verification:**
|
**Phase 1 Verification:**
|
||||||
- [ ] Cleaned up duplicate verification.go file
|
- [ ] Converted 18 CREATE TABLE → CREATE TABLE IF NOT EXISTS (27 total)
|
||||||
- [ ] Verified all CREATE TABLE use IF NOT EXISTS (36 total)
|
- [ ] Converted 62 CREATE INDEX → CREATE INDEX IF NOT EXISTS (82 total)
|
||||||
- [ ] Verified all CREATE INDEX use IF NOT EXISTS (102 total)
|
- [ ] Added ON CONFLICT to 2 INSERT statements (3 total)
|
||||||
- [ ] Verified all INSERT have ON CONFLICT (3 total)
|
|
||||||
- [ ] Verified schema.sql syntax is valid
|
- [ ] Verified schema.sql syntax is valid
|
||||||
|
- [ ] Confirmed single schema.sql exists as source of truth
|
||||||
|
|
||||||
**Phase 2 Verification:**
|
**Phase 2 Verification:**
|
||||||
- [ ] `internal/database/schema/schema.go` compiles without errors
|
- [ ] `internal/database/schema.go` compiles without errors
|
||||||
- [ ] Single cohesive package with all related functions
|
- [ ] Single file with all schema initialization functions
|
||||||
- [ ] Imports only what's used (embed, pgx v5)
|
- [ ] Imports only what's used (bufio, embed, pgx v5)
|
||||||
- [ ] Uses *pgxpool.Pool concrete type
|
- [ ] Uses *pgxpool.Pool concrete type
|
||||||
- [ ] Pre-computed constant for lock ID (582394759234)
|
- [ ] Pre-computed constant for lock ID (7804706162000639061)
|
||||||
- [ ] Stream scanning implementation for performance
|
- [ ] Stream scanning implementation for performance
|
||||||
- [ ] schema.sql successfully embedded and copied
|
- [ ] Successfully embeds existing schema.sql file
|
||||||
|
- [ ] No schema file duplication (single source of truth)
|
||||||
|
|
||||||
**Phase 3 Verification:**
|
**Phase 3 Verification:**
|
||||||
- [ ] `cmd/server/main.go` compiles without errors
|
- [ ] `cmd/server/main.go` compiles without errors
|
||||||
- [ ] Schema initialization call added before handler creation
|
- [ ] Schema initialization call added before handler creation
|
||||||
- [ ] Proper error handling with Fatal on failure
|
- [ ] Proper error handling with Fatal on failure
|
||||||
|
- [ ] Uses correct import path (`database.Initialize`)
|
||||||
|
|
||||||
**Integration Testing:**
|
**Integration Testing:**
|
||||||
- [ ] Fresh database initializes correctly
|
- [ ] Fresh database initializes correctly
|
||||||
@@ -425,18 +455,12 @@ func main() {
|
|||||||
|
|
||||||
If critical errors occur:
|
If critical errors occur:
|
||||||
|
|
||||||
### Option 1: Restore Single File
|
### Option 1: Restore Schema File
|
||||||
```bash
|
```bash
|
||||||
git checkout HEAD -- internal/database/schema/schema.go
|
git checkout HEAD -- internal/database/schema.go
|
||||||
```
|
```
|
||||||
|
|
||||||
### Option 2: Restore Package
|
### Option 2: Restore Integration
|
||||||
```bash
|
|
||||||
rm -rf internal/database/schema
|
|
||||||
git checkout HEAD -- internal/database/schema/
|
|
||||||
```
|
|
||||||
|
|
||||||
### Option 3: Restore Integration
|
|
||||||
```bash
|
```bash
|
||||||
git checkout HEAD -- cmd/server/main.go
|
git checkout HEAD -- cmd/server/main.go
|
||||||
```
|
```
|
||||||
@@ -458,7 +482,7 @@ The app will initialize the database automatically on first startup.
|
|||||||
|
|
||||||
**Monitor logs for:**
|
**Monitor logs for:**
|
||||||
```
|
```
|
||||||
✅ Database schema initialized and verified (36 tables verified)
|
✅ Database schema initialized and verified (27 tables verified)
|
||||||
```
|
```
|
||||||
|
|
||||||
### Subsequent Deployments
|
### Subsequent Deployments
|
||||||
@@ -494,7 +518,7 @@ podman exec bookhoard_db psql -U postgres -d bookhoard -c "\dt"
|
|||||||
### Check advisory locks:
|
### Check advisory locks:
|
||||||
```bash
|
```bash
|
||||||
podman exec bookhoard_db psql -U postgres -d bookhoard -c "
|
podman exec bookhoard_db psql -U postgres -d bookhoard -c "
|
||||||
SELECT * FROM pg_locks WHERE objid = 582394759234;
|
SELECT * FROM pg_locks WHERE objid = 7804706162000639061;
|
||||||
"
|
"
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -506,4 +530,12 @@ podman compose up -d
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 📚 Documentation
|
||||||
|
|
||||||
|
Create documentation explaining the automatic schema initialization:
|
||||||
|
- **docs/contributing/database-schema.md** - How schema initialization works, how to modify schema safely
|
||||||
|
- **README.md** - Add "Database Initialization" section documenting first-run behavior
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
**End of Refined Implementation Plan**
|
**End of Refined Implementation Plan**
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user