Files

T

john-okeefe 249085d942 docs: update implementation plan with API key authentication strategy

- Replace serial number approach with API key in URL path for Kobo
- Add authentication strategy section documenting Kobo and KOReader methods
- Update unified authentication architecture to support URL path parameters
- Document Komga-proven approach for stock Kobo firmware
- Update feature matrix with new authentication methods
- Revise user flows for API key-based registration
- Clarify OPDS security (already using DeviceAuthMiddleware)
- Update security considerations to reflect revocable API keys

2026-02-12 10:52:50 -05:00

46 KiB

Raw Blame History

Bookhoard Device Authentication & Sync Implementation Plan

Executive Summary

Current Problem:

OPDS routes are publicly accessible (security vulnerability)
Kobo sync requires custom authentication approach
KOReader documentation is incorrect about Basic Auth
No clear authentication strategy for different device types

Vision: Transform Bookhoard into a comprehensive Kindle-replacement ecosystem with seamless device sync, supporting both Kobo (native) and KOReader (via plugin) with full functionality including progress, highlights, notes, and bookmarks.

Authentication Strategy:

Kobo: API key in URL path (per-device, revocable, no jailbreak needed)
KOReader: Bearer token in Authorization header (per-device, revocable, via plugin)
OPDS: Both authentication methods supported

1. Current State Analysis

1.1 Authentication Landscape

Existing Infrastructure:

// DeviceAuthMiddleware - Currently only supports Bearer tokens
// File: internal/middleware/device_auth.go:37-108
func (m *DeviceAuthMiddleware) Authenticate(next echo.HandlerFunc) echo.HandlerFunc {
    // Only checks Authorization: Bearer {token}
    // Validates against devices.auth_token
}

Routes Using Device Auth:

/api/sync/kobo/* - Kobo sync endpoints
/api/sync/koreader/* - KOReader sync endpoints
/opds/devices/* - OPDS catalog (currently NOT protected, needs auth)

Database Schema:

-- devices table has auth_token field for all device authentication
-- Can store revocable API keys for Kobo devices

1.2 Kobo Integration Status

Proven Approach (Komga):

Komga successfully handles Kobo sync without jailbreak using API keys in URL path
Configuration: api_endpoint=https://komga.example.com/kobo/{api_key}
Kobo firmware supports custom api_endpoint in Kobo eReader.conf
API key embedded in URL path works reliably with stock Kobo firmware

What's Currently Broken:

Current middleware expects Bearer token in Authorization header
Routes need to support API key in URL path parameter
Device auth tokens (API keys) already generated during registration

What's Working:

Kobo sync handler implementation exists and is functional
Endpoints registered: /api/sync/kobo/markup, /bookmark, /v1/initialization, etc.
Device auth tokens (API keys) already generated during registration
Can sync: progress, highlights, notes, bookmarks once auth is fixed

1.3 KOReader Integration Status

What's Broken:

Documentation says use "Basic Auth" - this is WRONG
KOReader doesn't support custom headers natively for sync
Bearer token can't be sent via standard KOReader settings

What's Working:

KOReader sync handler supports full feature set
Can sync: progress, bookmarks, highlights, notes, library metadata
Plugin architecture exists but no Bookhoard plugin yet

1.4 OPDS Status

Current:

Routes registered in internal/router/opds.go:11-12
✅ Already using DeviceAuthMiddleware (line 12: opds.Use(cfg.DeviceAuthMiddleware.Authenticate))
✅ Authentication is required - not publicly accessible
Routes: /opds/devices/:deviceId/* - all protected

Required:

✅ No changes needed for OPDS security
After Phase 1, OPDS will automatically support both auth methods:
- Kobo: API key in URL path (?token= or path parameter)
- KOReader: Bearer token via Authorization header

2. User Priorities (Ranked)

Priority 1: Functionality First

User Quote: "I want a Kindle system replacement, otherwise I could just use Booklore or KOReader"

Requirements:

✅ Full sync: progress, highlights, notes, bookmarks
✅ Works on both Kobo and KOReader
✅ Seamless user experience
✅ OPDS for wireless book delivery
✅ Two-way sync (device ↔ server)

Priority 2: Security Second (with documentation)

User Quote: "This is a self-hosted app so as long as the user is aware of the security implications in the documentation it should be fine"

Acceptable Trade-offs:

API key in URL for Kobo (per-device, revocable, documented security considerations)
Token-based for KOReader (more secure)
Users responsible for network security (VPN, local network, HTTPS)
Security implications clearly documented

Priority 3: Simple Setup

Goal: Minimal friction for users

Kobo Experience:

Register device in Bookhoard UI
Copy device API key from device management page
One line in Kobo eReader.conf with API key
Token can be regenerated if compromised
Works immediately with stock firmware

KOReader Experience:

Copy plugin files
Enter server URL and token
One-time setup
Background sync works automatically

3. Target State

3.1 Unified Authentication Architecture

┌─────────────────────────────────────────────────────────────────┐
│              DeviceAuthMiddleware (Enhanced)                   │
│  Unified authentication using revocable API keys:               │
│                                                               │
│  1. Authorization: Bearer {token}                             │
│     → Extract from Authorization header                        │
│     → Lookup device by auth_token                             │
│     → Used by: KOReader, OPDS apps, API clients              │
│                                                               │
│  2. URL Path Parameter: /api/sync/kobo/{token}                │
│     → Extract token from URL path (c.Param("token"))          │
│     → Lookup device by auth_token                             │
│     → Used by: Kobo e-readers (stock firmware, no jailbreak) │
│                                                               │
│  3. Query Parameter: ?token={token}                           │
│     → Extract from query string (c.QueryParam("token"))       │
│     → Lookup device by auth_token                             │
│     → Used by: OPDS catalog access                           │
│                                                               │
│  All methods result in: device context set in echo.Context    │
└─────────────────────────────────────────────────────────────────┘

Why This Architecture:

Simple: All methods use same auth_token lookup (no new database logic)
Secure: API keys are random UUIDs, revocable, can be regenerated
Universal: All devices get auth_tokens (existing field, no schema change)
Proven: Komga uses this exact approach for Kobo sync successfully
No Firmware Limitations: URL path parameter works with stock Kobo firmware (no jailbreak needed)
Revocable: API keys can be regenerated by users without device re-registration
Single Authentication Method: All device types use the same auth_token lookup, simplifying codebase


### 3.2 Feature Matrix

| Feature | Kobo (Stock Firmware) | KOReader (Plugin) |
|---------|---------------|-------------------|
| **Progress Sync** | ✅ Automatic (API key in URL) | ✅ Automatic (Bearer token) |
| **Highlights** | ✅ Native support | ✅ Via plugin API |
| **Bookmarks** | ✅ Native support | ✅ Via plugin API |
| **Notes** | ✅ Native support | ✅ Via plugin API |
| **OPDS** | ✅ Native support | ✅ Native OPDS support |
| **Setup Complexity** | Low (1 config line) | Medium (plugin install) |
| **Auth Method** | API key in URL path | Bearer token in header |
| **Security** | High (revocable tokens) | High (revocable tokens) |

### 3.3 User Flows

**Kobo User Journey:**
1. Open Bookhoard web UI
2. Navigate to Device Management → Add New Device
3. Enter device name and select "Kobo" as device type
4. Click Register Device
5. From device details page, copy full Kobo Sync URL with API key:
    ```
    http://YOUR_IP:8765/api/sync/kobo/YOUR_API_KEY
    ```
6. Connect Kobo to computer, open `.kobo/Kobo/Kobo eReader.conf`
7. Add/edit line: `api_endpoint=http://YOUR_IP:8765/api/sync/kobo/YOUR_API_KEY`
    - Paste the entire URL from step 5 (includes device API key)
8. Save and eject Kobo (restarts automatically)
9. Kobo syncs to Bookhoard instead of Kobo store
10. For OPDS: Add catalog URL with `?token=YOUR_API_KEY` parameter

**Important Note - Token Regeneration:**
- If token is compromised or lost, regenerate from device management page
- OLD token will immediately stop working
- User must update `api_endpoint` line with NEW token
- One-click copy button makes this easy

**KOReader User Journey:**
1. Install KOReader on device
2. Open Bookhoard web UI
3. Register device → copy API key
4. Approve device
5. Install Bookhoard plugin → paste API key
6. Configure sync settings
7. Read books → automatic sync
8. Add OPDS catalog with API key in header (via plugin)

---

## 4. Implementation Plan

### Phase 1: Enhanced Authentication (Week 1)
**Goal:** Make auth work for both Kobo (API key in URL) and KOReader (Bearer token)

**Tasks:**

1. **Update Routing for Kobo** (`internal/router/sync.go`)
    - [ ] Change Kobo route group from `/api/sync/kobo` to `/api/sync/kobo/:token`
    - [ ] Update all Kobo route handlers to use path parameter
    - [ ] Test that routes still work with Bearer token (backward compatibility)

2. **Update DeviceAuthMiddleware** (`internal/middleware/device_auth.go`)
    - [ ] Try Bearer token lookup first (existing behavior)
    - [ ] If no Bearer token, check for `token` path parameter (`c.Param("token")`)
    - [ ] Query database: `GetDeviceByAuthToken(token)` (already exists)
    - [ ] Set device context on successful auth
    - [ ] Return 401 if both methods fail

3. **Add Token Regeneration Backend** (`internal/handlers/devices.go`)
    - [ ] Add endpoint: `POST /api/devices/:id/regenerate-token`
    - [ ] Generate new random UUID for auth_token
    - [ ] Update device in database
    - [ ] Return new token to user
    - [ ] Require JWT auth (user must be logged in)

4. **Update Device Registration UI** (`templates/devices.templ`, `templates/partials/device-management.templ`)
   - [ ] **Show "Copy Full Sync URL" button for each device** (uses clipboard API)
   - [ ] For Kobo: Display complete URL: `http://IP:8765/api/sync/kobo/{DEVICE_TOKEN}`
   - [ ] **Add "Regenerate Token" button** (with confirmation dialog: "This will revoke current token. Continue?")
   - [ ] After regeneration, show toast: "Token regenerated - update Kobo config"
   - [ ] For KOReader: Show auth token in copyable input field
   - [ ] Display clear setup instructions per device type with full URL example

5. **Apply Middleware to OPDS** (`internal/router/opds.go`)
    - [ ] Verify OPDS routes have DeviceAuthMiddleware (already applied)
    - [ ] Add support for `?token=` query parameter for OPDS access
    - [ ] Update comments to reflect dual auth support

**Testing:**
- Kobo device sync with api-key url auth
- KOReader sync with Bearer token auth
- OPDS access with both auth methods

---

### Phase 2: Kobo Integration (Week 1-2)
**Goal:** Seamless Kobo experience with API key-based auth (stock firmware)

**Tasks:**

1. **Kobo Setup Documentation** (`docs/user/devices/kobo-setup.md`)
    - [ ] Show how to get API key from Bookhoard device management page
    - [ ] Step-by-step guide for editing `Kobo eReader.conf`
    - [ ] Include example config: `api_endpoint=http://192.168.1.100:8765/api/sync/kobo/{API_KEY}`
    - [ ] Security warning about API keys in logs
    - [ ] Network security recommendations (VPN, local network)
    - [ ] Instructions for regenerating token if compromised

2. **Kobo Sync Validation** (`cmd/server/tests/kobo_test.go`)
    - [ ] Update tests to use API key in URL path
    - [ ] Test sync endpoints with `/api/sync/kobo/{token}/markup` format
    - [ ] Verify all features work: progress, highlights, bookmarks, notes
    - [ ] Test token regeneration doesn't break existing sync

3. **Kobo OPDS Integration**
    - [ ] Document OPDS URL format: `http://IP:8765/opds/devices/{DEVICE_ID}/catalog?token={API_KEY}`
    - [ ] Test OPDS browsing with API key in URL parameter
    - [ ] Test book downloads

**Success Criteria:**
- New Kobo device registered and syncing within 5 minutes
- All sync features working without jailbreak
- API key can be regenerated if compromised
- Security implications clearly documented
- Works with stock Kobo firmware (no modifications)

---

### Phase 3: KOReader Plugin Development (Week 2-4)
**Goal:** Full-featured KOReader plugin with Bearer token auth

**Tasks:**

1. **Plugin Architecture**
   - [ ] Create `koreader-plugin/bookhoard.koplugin/` directory
   - [ ] Implement `_meta.lua` (plugin metadata)
   - [ ] Implement `main.lua` (entry point)
   
2. **Core Plugin Features**
   - [ ] Settings UI: server URL, auth token, sync options
   - [ ] HTTP client with Bearer token authentication
   - [ ] Auto-sync on events:
     - Page turn (configurable: every N pages)
     - Bookmark added/deleted
     - Highlight created/deleted
     - Note added/edited
   - [ ] Manual sync button in KOReader menu
   - [ ] Sync status indicator
   - [ ] Error handling and retry logic

3. **Plugin API Integration**
   - [ ] Call `/api/sync/koreader/progress` with reading position
   - [ ] Call `/api/sync/koreader/bookmarks` with annotations
   - [ ] Call `/api/sync/koreader/metadata` to get server-side progress
   - [ ] Handle conflicts (server vs local progress)

4. **Plugin Documentation**
   - [ ] Installation guide (copy files, restart KOReader)
   - [ ] Configuration guide (get token from Bookhoard UI)
   - [ ] Usage instructions
   - [ ] Troubleshooting guide

**Success Criteria:**
- Plugin installed and configured in under 10 minutes
- Background sync works transparently
- All annotation types sync properly
- No Lua errors or crashes

---

### Phase 4: OPDS Security & Integration (Week 2-3)
**Goal:** Secure OPDS with unified device authentication

**Tasks:**

1. **Secure OPDS Routes** (`internal/router/opds.go`)
   - [ ] Apply `DeviceAuthMiddleware` to all OPDS endpoints
   - [ ] Ensure auth works with both api-key and Bearer methods
   - [ ] Update router comments

2. **Kobo OPDS Access**
   - [ ] Document OPDS URL with device ID
   - [ ] Test OPDS access via Kobo browser
   - [ ] Test book downloads

3. **KOReader OPDS Access**
   - [ ] Works natively via KOReader's OPDS support
   - [ ] Uses Bearer token in header
   - [ ] Document catalog URL format

4. **Test Coverage** (`cmd/server/tests/opds_test.go`)
   - [ ] Test unauthenticated access → expect 401
   - [ ] Test with valid Bearer token → expect 200
   - [ ] Test with valid api-key url → expect 200
   - [ ] Test book download with auth

---

### Phase 5: Testing & Documentation (Week 4)
**Goal:** Production-ready release with comprehensive docs

**Tasks:**

1. **End-to-End Testing**
   - [ ] Fresh Kobo setup: register → sync → highlights → OPDS
   - [ ] Fresh KOReader setup: install plugin → sync → highlights → OPDS
   - [ ] Cross-device sync: Kobo ↔ KOReader same book
   - [ ] Offline scenarios and conflict resolution
   - [ ] Error handling and recovery

2. **Security Documentation**
   - [ ] Security implications page
   - [ ] Network security recommendations
    - [ ] Comparison: API Key (URL path) vs Bearer token - both use revocable auth_token
   - [ ] Threat model for self-hosted users

3. **User Documentation**
   - [ ] Quick start guide (Kobo)
   - [ ] Quick start guide (KOReader)
   - [ ] Feature comparison table
   - [ ] FAQ and troubleshooting

4. **Release Preparation**
   - [ ] Version bump
   - [ ] Changelog
   - [ ] Migration guide (if needed)

---

## 5. Technical Decisions

### 5.1 Why API Key in URL Path for Kobo?

**Approach Chosen:** Per-device API key embedded in URL path: `/api/sync/kobo/{api_key}/...`

**Pros:**
- No jailbreak needed - works with stock Kobo firmware
- Proven approach - Komga uses this successfully
- API keys are random UUIDs (cryptographically secure)
- Tokens can be revoked and regenerated
- Single configuration line in Kobo config
- Simple token management (user can regenerate)
- Uses existing `auth_token` field (no schema change)

**Cons:**
- API key visible in Kobo logs and server logs
- URL-based authentication (less secure than headers)
- Key appears in Kobo config file (plain text)

**Mitigations:**
- Use HTTPS for sync (reverse proxy)
- Document that API keys are sensitive
- Recommend local network only (VPN for remote)
- Provide one-click token regeneration
- Acceptable security for self-hosted use (like Komga)

**Alternative Considered:**
- **Serial-based auth**: Rejected because it uses predictable serials that can't be revoked and is less secure
- **Kobo store token**: Rejected because it's managed by Kobo, not under our control
- **Custom HTTP headers**: Rejected because Kobo firmware doesn't support custom headers

**Authentication Strategy:**
- Kobo: API key in URL path (per-device, revocable)
- KOReader: Bearer token (per-device, revocable)
- OPDS: Both methods supported (API key via query parameter, bearer token via header)

### 5.2 Why Not Use Kobo's Token?

Kobo sends `Authorization: Bearer {kobo-store-token}` which is:
- Managed by Kobo (not under our control)
- May rotate/expire without warning
- Only valid for Kobo's servers
- Doesn't identify device in our system
- Can't be extracted from Kobo config without jailbreak

**Decision:** Use API key in URL path instead (proven by Komga). The API key in URL path provides per-device authentication that is revocable and works with stock Kobo firmware without jailbreak.

### 5.3 Token Regeneration Strategy

**Backend:**
- Endpoint: `POST /api/devices/:id/regenerate-token`
- Requires: User JWT auth (device owner)
- Action: Generate new UUID, update `devices.auth_token`
- Response: Return new token to user

**Frontend:**
- "Regenerate Token" button on device management page
- Confirmation dialog: "This will revoke current token. Continue?"
- One-click copy to clipboard
- Update display instructions with new token
- Toast notification: "Token regenerated successfully"

**Impact:**
- Old token immediately invalid
- Device must update config with new token
- Sync will fail until config updated
- Useful for: Lost tokens, compromised devices, periodic rotation

### 5.4 Why Bearer Token for KOReader?

**Pros:**
- Cryptographically secure (random UUID)
- Can be revoked/regenerated
- Standard authentication method
- Works with KOReader plugin architecture
- Plugin can store token securely

**Cons:**
- Requires plugin (can't use native settings)
- Token must be copy-pasted during setup
- User must manage token

**Trade-off Accepted:**
More secure but requires plugin installation. Justified because KOReader users are typically more technical and willing to install plugins.

**Note:** Same token regeneration as Kobo (reuses backend logic)

---

## 6. Files to Modify

### Core Authentication
- `internal/middleware/device_auth.go` - Add URL path token extraction
- `internal/router/sync.go` - Update Kobo routes to use path parameter
- `internal/router/opds.go` - Add ?token= query parameter support (already has middleware)

### Device Management
- `internal/handlers/devices.go` - Add regenerate token endpoint
- `internal/database/queries/queries.sql` - No changes needed (auth_token already exists)
- `templates/devices.templ` - Add copy token, regenerate token buttons
- `templates/partials/device-management.templ` - Kobo config instructions with API key

### Testing
- `cmd/server/tests/kobo_test.go` - Update for URL path token
- `cmd/server/tests/opds_test.go` - Add ?token= auth tests
- `cmd/server/tests/test_helpers.go` - Add token regeneration helpers

### Documentation
- `docs/user/devices/kobo-setup.md` - Rewrite for API key auth
- `docs/user/devices/koreader-setup.md` - Update for Bearer token
- `docs/user/devices/security.md` - New security considerations doc
- `docs/developer/api/kobo/` - Update API docs with path parameters

### New Files
- `koreader-plugin/bookhoard.koplugin/_meta.lua` - Plugin metadata (Phase 3)
- `koreader-plugin/bookhoard.koplugin/main.lua` - Plugin implementation (Phase 3)
- `docs/user/devices/security.md` - Security considerations

---

## 7. Success Metrics

**Functionality:**
- ✅ Kobo syncs progress, highlights, notes, bookmarks automatically
- ✅ KOReader syncs same features via plugin
- ✅ OPDS works on both platforms
- ✅ Cross-device sync works (read on Kobo, continue on KOReader)

**Usability:**
- ✅ Kobo setup time: < 5 minutes
- ✅ KOReader setup time: < 10 minutes
- ✅ No token management for Kobo users
- ✅ Clear documentation with security warnings

**Security:**
- ✅ OPDS no longer publicly accessible
- ✅ Auth required for all sync endpoints
- ✅ Security implications documented
- ✅ Network security recommendations provided

---

## 8. Future Enhancements (Out of Scope)

- Auto-discovery of devices on network
- QR code setup for mobile devices
- Background sync service (no plugin needed)
- Readwise/Joplin integration
- Calibre integration
- Mobile apps (iOS/Android)

---

## 9. Decisions Made (User Input)

Based on user feedback, the following decisions have been finalized:

### 9.1 KOReader Sync Trigger
**Decision:** Configurable with multiple options
- **Auto-sync**: Every N pages (configurable, default every 10 pages)
- **Auto-sync**: Every N minutes (configurable, default every 5 minutes)
- **Manual sync**: Always available via menu
- **Event-triggered**: On bookmark/highlight/note creation (immediate)

**Rationale:** User wants automatic sync for convenience but manual fallback for control.

### 9.2 Plugin Distribution
**Decision:** Separate repository under Bookhoard organization
- Repository: `github.com/bookhoard/koreader-plugin`
- Licensed under same terms as Bookhoard
- Versioned independently
- Referenced in main Bookhoard documentation

**Rationale:** Clean separation of concerns, easier plugin-specific issues/PRs.

### 9.3 Kobo Authentication
**Decision:** API key in URL path (per-device)
- No serial number needed
- Token entered in device registration (already exists)
- Configured in Kobo as: `api_endpoint=http://IP:8765/api/sync/kobo/{TOKEN}`
- Token can be regenerated by user (new endpoint)
- Stored in `devices.auth_token` field (no schema change)

**Rationale:** Simplest user experience (just copy token), most secure (revocable), proven to work (Komga).

### 9.4 Conflict Resolution
**Decision:** Use existing codebase strategy
- Already implemented in `internal/handlers/koreader.go`
- Uses timestamp-based last-write-wins
- Device priority can be configured per-device

**Rationale:** Don't reinvent the wheel, existing implementation is sufficient.

### 9.5 Implementation Order
**Decision:** Parallel implementation with separate sections
- Phase 2a: Kobo Integration
- Phase 2b: KOReader Plugin (runs concurrently)
- Separate milestones and deliverables
- Can ship Kobo support before plugin is ready

**Rationale:** Faster time-to-market for Kobo users, plugin can follow.

### 9.6 KOReader Device Identification Strategy

**Problem:** Multiple KOReader installations each register as separate devices, exhausting `max_devices` limit even though they represent the same physical device.

**Note:** This is about device MANAGEMENT, not authentication. Authentication uses `auth_token` (API keys) for all devices. Device identifier is for tracking multiple installations of the same physical device.

**Decision:** Self-registration with plugin-generated device ID that persists across KOReader reinstalls.

**Architecture:**
- Plugin generates unique `device_id` on first launch (stored in KOReader settings)
- Plugin sends `device_identifier` during registration API call
- Backend validates `auth_token` matches device record
- Reinstalling KOReader reads same `device_id` from settings, reuses existing device record

**Registration Flow Options:**

| Approach | User Experience | Complexity | Pros/Cons |
|----------|-----------------|------------|-------------|
| **A. Manual Link** | User registers in Bookhoard, copies Device ID + Token | Low | ✅ Simple server<br>❌ User copies two values |
| **B. Self-Register** | Plugin auto-registers, user approves in web UI | Medium | ✅ One-time setup<br>⚠️ Requires approval endpoint |
| **C. QR Code Bridge** | Scan QR code with Device ID + Token | Low-Medium | ✅ Very user-friendly<br>❌ Requires QR library |

**Recommendation:** **Option B (Self-Register)** for best UX.

**Backend Changes Required:**
```go
// Devices table - Add Device Identifier (Already Exists!)
-- devices.device_identifier already exists (models.go:67)
-- No schema change needed - just use it properly

// internal/handlers/devices.go - InitiateDeviceRegistration()
// Add optional pre_generated_device_id parameter:
DeviceIdentifier string `json:"device_identifier" validate:"omitempty,uuid"`

Authentication Middleware Update:

// Try Bearer token first (KOReader, API clients)
// Then try URL path parameter (Kobo, OPDS)
func (m *DeviceAuthMiddleware) Authenticate(next echo.HandlerFunc) echo.HandlerFunc {
    // 1. Try Bearer token header (KOReader, API clients)
    authHeader := c.Request().Header.Get("Authorization")
    if authHeader != "" {
        token := strings.TrimPrefix(authHeader, "Bearer ")
        device, err := m.db.GetDeviceByAuthToken(c.Request().Context(), token)
        if err == nil {
            return m.setDeviceContext(c, device)
        }
    }

    // 2. Try URL path parameter (Kobo, OPDS)
    urlToken := c.Param("token")
    if urlToken == "" {
        urlToken = c.QueryParam("token")  // Fallback to query param
    }
    if urlToken != "" {
        device, err := m.db.GetDeviceByAuthToken(c.Request().Context(), urlToken)
        if err == nil {
            return m.setDeviceContext(c, device)
        }
    }

    return c.JSON(401, map[string]string{"error": "authentication required"})
}

Plugin Changes Required:

-- First Run Detection:
-- On first launch:
if not G_Settings:hasSetting("bookhoard_device_id") then
    local device_id = uuid.generate()
    G_Settings:saveSetting("bookhoard_device_id", device_id)
    G_Settings:saveSetting("bookhoard_device_name", "KOReader on " .. Device.model)
    
    -- Trigger registration
    register_device()
end

-- Registration API Call:
-- POST /api/devices/register
{
    device_name: "KOReader on Kindle Paperwhite",
    device_type: "koreader",
    device_identifier: "550e8400-e29b..."  -- From plugin settings
}

Device Management UI:

Display device_identifier for each device
"Copy Device ID" button for KOReader setup
"Regenerate Token" doesn't change Device ID
Show "Linked Devices" count (e.g., "3 KOReader installations linked")

Sync Flow With Device ID:

-- Plugin Headers:
Authorization: Bearer {auth_token}
X-Bookhoard-Device-ID: {device_identifier}

// All devices use auth_token for authentication (single method)
func (m *DeviceAuthMiddleware) Authenticate(next echo.HandlerFunc) echo.HandlerFunc {
    // 1. Try Bearer token header (KOReader, API clients)
    authHeader := c.Request().Header.Get("Authorization")
    if authHeader != "" {
        token := strings.TrimPrefix(authHeader, "Bearer ")
        device, err := m.db.GetDeviceByAuthToken(c.Request().Context(), token)
        if err == nil {
            return m.setDeviceContext(c, device)
        }
    }

    // 2. Try URL path parameter (Kobo, OPDS)
    urlToken := c.Param("token")
    if urlToken == "" {
        urlToken = c.QueryParam("token")  // Fallback to query param
    }
    if urlToken != "" {
        device, err := m.db.GetDeviceByAuthToken(c.Request().Context(), urlToken)
        if err == nil {
            return m.setDeviceContext(c, device)
        }
    }

    return c.JSON(401, map[string]string{"error": "authentication required"})
}

Benefits:

Aspect	Current	With Device ID
Reinstalls	New device each time	Same device, update token only
max_devices	Counts installs, not devices	Accurate device count
Security	Token-only	Device ID + token (two-factor)
Troubleshooting	"Which KOReader is this?"	Clear device identification
Plugin Transfer	Manual token copy	Migrate settings file

Phase Integration: Implement Device ID in Phase 1 (alongside dual authentication). It's same amount of work but solves max_devices problem elegantly.

Questions for User:

Registration Workflow: Should KOReader plugin self-register (user approves in browser) or require manual registration first?
Device ID Persistence: Where does KOReader store settings?
- /mnt/us/koreader/settings.bookhoard.lua?
- System G_Settings?
- This determines if Device ID survives plugin updates
max_devices Behavior:
- Should each Device ID count as 1 device (recommended)?
- Or limit tokens per Device ID (more complex)?
Backward Compatibility:
- How to handle existing KOReader installations without Device ID?
- Force re-registration? Auto-migrate?
KOReader Plugin Repo:
- Does github.com/bookhoard/koreader-plugin exist?
- Should Device ID generation happen there, or in main Bookhoard repo?

10. Historical Context & Conversation Summary

Why This Plan Exists

This implementation plan emerged from a detailed analysis of the current authentication and sync architecture. Key discoveries from codebase review:

The Authentication Problem

Original Issue: The plan started with @fix-opds-device-authentication.md which identified:

OPDS routes were publicly accessible (security vulnerability)
Test GetDeviceCatalog_WithoutDeviceAuth expected 401 but got 404
DeviceAuthMiddleware was already applied to OPDS routes, but tests were failing

Root Cause Discovery:

Kobo devices send Authorization: Bearer {kobo-store-token} (Kobo's store token, not Bookhoard's)
Kobo cannot send custom Bearer tokens through eReader.conf
Current middleware only supports Bearer tokens in Authorization header
Solution: Use API key in URL path (proven by Komga)

The Sidecar File Red Herring

Initial Thought: Use .bookhoard.json sidecar file to pass device tokens to Kobo

Problem Discovered:

Sidecar handler exists (internal/handlers/sidecar.go) but routes are NOT registered
Kobo firmware is locked down - cannot read custom config files
No plugin architecture on Kobo (unlike KOReader)
Would require Kobo firmware modification (impossible for most users)

Decision: Abandon sidecar approach, use API key in URL path (proven by Komga)

API keys are revocable and regenerable
Works with stock Kobo firmware
Simple user configuration

KOReader Documentation Bug

Original Documentation: Claimed KOReader uses "Basic Auth" with username/password

Reality:

KOReader sync sends Authorization: Bearer {token} to sync servers
The official KOReader sync protocol uses MD5 hashed passwords
Bookhoard's implementation expects Bearer tokens in DeviceAuthMiddleware
KOReader cannot send custom headers through native settings

Solution: Create KOReader plugin that can send proper Bearer tokens

Booklore Analysis

Investigation: Analyzed how Booklore handles Kobo authentication

Booklore Approach:

Token embedded in URL path: /api/kobo/{token}/...
Works but token appears in logs
One token per user (not per device)
OPDS uses separate Basic Auth

Decision: Don't copy Booklore - less secure. Use device-specific auth with proper tokens for KOReader.

Plugin Feasibility Research

Concern: Is KOReader a "moving target" for plugin development?

Findings:

UI plugins are unstable (Issue #13942) - frequent breaking changes
Backend/sync plugins (like Wallabag) are stable
Wallabag2 plugin has minimal updates over years
HTTP client APIs don't change often

Decision: Proceed with plugin - sync plugins are low-risk

Technical Constraints Discovered

Kobo Firmware Limitations:
- Cannot send custom HTTP headers
- Cannot install plugins
- Settings in eReader.conf are limited to specific keys
- Solution: API key in URL path works with stock firmware
KOReader Capabilities:
- Can install plugins (copy files to /koreader/plugins/)
- Lua-based plugin architecture
- Can send custom HTTP requests
- Plugin can register itself
- Native OPDS client support
Security Trade-offs:
- API Key in URl: Which means it's stored in logs
- Bearer tokens: Cryptographically secure, requires setup
- User accepted security/usability trade-off for self-hosted use

Why Functionality Trumps Security

User Quote: "I want a Kindle system replacement, otherwise I could just use booklore or KOReader. My biggest concern is do any of these limit any of the syncing/downloading implementation I already have. Functionality and features are more important than anything else. Although security does come in at a close second."

Translation:

Self-hosted = trusted network environment
User understands and accepts risks
Willing to document security implications
Prioritizes working features over perfect security

Resulting Architecture:

Kobo: API key in URL path (revocable, simple)
KOReader: Bearer token (high security, requires plugin setup)
Both get full feature parity
Security implications clearly documented

11. Updated Implementation Timeline

Parallel Track Structure

Track A: Kobo Integration

Phase 1: Enhanced Auth (Week 1)
Phase 2a: Kobo Implementation (Week 1-2)
Phase 4: Kobo Testing (Week 3)

Track B: KOReader Plugin

Phase 1: Enhanced Auth (Week 1) [shared]
Phase 2b: Plugin Development (Week 2-4)
Phase 4: Plugin Testing (Week 4)

Track C: OPDS Security

Phase 3: OPDS Integration (Week 2-3)

Track D: Documentation

Phase 5: Final Docs & Release (Week 4)

Milestones

Milestone 1 (End of Week 2): Kobo Fully Functional

Kobo: API key in URL (works with stock firmware)
All sync features operational
OPDS access secured
Can ship to Kobo users

Milestone 2 (End of Week 4): KOReader Plugin Ready

Plugin released in separate repo
Full feature parity with Kobo
Documentation complete
Production release

12. Files & Components Reference

Critical Files for Implementation

Authentication Core:

internal/middleware/device_auth.go:37-108 - Main auth middleware
internal/database/queries/queries.sql - Device queries
internal/database/models.go - Device model (has device_identifier)

Routing:

internal/router/opds.go:11-12 - OPDS routes (need middleware)
internal/router/sync.go:17-50 - Sync routes
internal/router/device.go - Device management routes

Handlers:

internal/handlers/kobo.go:26-679 - Kobo sync implementation
internal/handlers/koreader.go:23-897 - KOReader sync implementation
internal/handlers/opds.go - OPDS handler

Tests:

cmd/server/tests/kobo_test.go - Kobo tests
cmd/server/tests/opds_test.go - OPDS tests
cmd/server/tests/test_helpers.go - Test utilities

Documentation (to update):

docs/user/devices/kobo-setup.md - Currently has wrong info
docs/user/devices/koreader-setup.md - Needs Bearer token info

Sidecar (reference only, not implementing):

internal/handlers/sidecar.go - Exists but not wired up
Routes not registered in router

13. Next Steps

Immediate Actions (When Ready to Proceed)

Review this plan - Ensure all decisions and context are captured
Create KOReader plugin repo - Set up github.com/bookhoard/koreader-plugin
Begin Phase 1 - Enhanced DeviceAuthMiddleware
Parallel development - Kobo and KOReader tracks

Success Criteria (Reiterated)

Functional:

✅ Kobo: Enter serial → register → sync works immediately
✅ KOReader: Install plugin → plugin registers → sync works
✅ Both: Full sync (progress, highlights, notes, bookmarks)
✅ OPDS: Authenticated access on both platforms
✅ Cross-device: Read on one, continue on another

Security:

✅ OPDS no longer publicly accessible (authenticated with API key or Bearer token)
✅ All sync endpoints require device authentication (API key or bearer token)
✅ Device authentication uses revocable API keys (Kobo/KOReader)
✅ Security implications documented
✅ Network security recommendations provided

User Experience:

✅ Kobo setup: < 5 minutes
✅ KOReader setup: < 10 minutes
✅ Clear documentation
✅ Working examples

Plan Status: Ready for implementation Last Updated: Based on conversation ending with user decisions Note: User indicated it's late and not proceeding tonight

Ready to implement when you are. This plan captures all our discussion and decisions.

10. Risk Assessment

Low Risk:

API key in URL path (well-understood pattern)
OPDS security (straightforward middleware application)
Kobo sync (already implemented, just fixing auth route)

Medium Risk:

KOReader plugin development (Lua learning curve, potential API changes)
Cross-device sync conflicts (need clear resolution strategy)

High Risk:

KOReader API instability (if they change plugin APIs frequently)
User adoption (may resist plugin installation)

Mitigations:

Start with Kobo to prove concept
Simple plugin architecture (minimize breaking changes impact)
Excellent documentation to reduce friction
Community feedback loop

11. Timeline Summary

Phase	Duration	Key Deliverable
Phase 1	Week 1	Enhanced auth middleware working
Phase 2	Week 1-2	Kobo fully functional
Phase 3	Week 2-4	KOReader plugin complete
Phase 4	Week 2-3	OPDS secured
Phase 5	Week 4	Testing & docs complete

Total Duration: 4 weeks

Next Steps:

User reviews and approves plan
Answer open questions (Section 9)
Begin Phase 1 implementation
Weekly check-ins on progress

Ready to proceed?

14. Authentication Strategy Confirmation

14.1 Kobo Authentication: API Key in URL Path

Decision: Use per-device API keys embedded in URL path (proven by Komga)

Architecture:

Kobo route: /api/sync/kobo/{api_key}/*
API key generated during device registration
User copies full URL from device management page
Paste into Kobo's api_endpoint configuration

Why This Approach:

✅ Proven by Komga (production-tested, works reliably)
✅ No jailbreak needed - works with stock Kobo firmware
✅ API keys are revocable and can be regenerated
✅ Uses existing auth_token field (no schema change)
✅ Simple user configuration (one line in config file)
✅ More secure than serial-based authentication

Rejection: Serial-based via X-Kobo-Device Header

Serials are predictable and cannot be revoked
More complex implementation (JSON parsing, device_identifier lookup)
Less secure than random API keys
No benefit over URL path approach

14.2 Router Configuration

Required Change:

// Current routing
koboSync := e.Group("/api/sync/kobo")
koboSync.POST("/markup", cfg.DeviceAuthMiddleware.Authenticate(...))

// New routing (supports API key in URL path)
koboSync := e.Group("/api/sync/kobo/:token")
koboSync.POST("/markup", cfg.DeviceAuthMiddleware.Authenticate(...))

Middleware Enhancement:

// Add path parameter extraction (supports both Bearer token and URL token)
func (m *DeviceAuthMiddleware) Authenticate(next echo.HandlerFunc) echo.HandlerFunc {
    // 1. Try Bearer token header (KOReader, API clients)
    authHeader := c.Request().Header.Get("Authorization")
    if authHeader != "" {
        token := strings.TrimPrefix(authHeader, "Bearer ")
        device, err := m.db.GetDeviceByAuthToken(c.Request().Context(), token)
        if err == nil {
            return m.setDeviceContext(c, device)
        }
    }

    // 2. Try URL path parameter (Kobo, OPDS)
    urlToken := c.Param("token")
    if urlToken == "" {
        urlToken = c.QueryParam("token")  // Fallback to query param
    }
    if urlToken != "" {
        device, err := m.db.GetDeviceByAuthToken(c.Request().Context(), urlToken)
        if err == nil {
            return m.setDeviceContext(c, device)
        }
    }

    return c.JSON(401, map[string]string{"error": "authentication required"})
}

Frontend:

Display auth_token for each device (API key for Kobo, token for KOReader)
"Copy Device ID" button for KOReader setup
"Regenerate Token" doesn't change auth_token

14.3 Token Regeneration

Required Addition: Backend endpoint to regenerate tokens

Implementation:

// internal/handlers/devices.go
func (h *Handlers) RegenerateDeviceToken(c echo.Context) error {
    deviceID, err := uuid.Parse(c.Param("id"))
    // ... user auth check ...

    newToken := fmt.Sprintf("dev_%s", uuid.New().String())

    _, err = h.db.UpdateDevice(c.Request().Context(), database.UpdateDeviceParams{
        ID: pgtype.UUID{Bytes: deviceID.Bytes(), Valid: true},
        AuthToken: pgtype.Text{String: newToken, Valid: true},
    })

    return c.JSON(200, map[string]string{
        "auth_token": newToken,
        "message": "Token regenerated successfully",
    })
}

Route:

// internal/router/device.go
devices.PUT("/:id/regenerate-token", jwtMiddleware, h.RegenerateDeviceToken)